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Resume en frangais 


Introduction 

De nombreux systemes critiques doivent satisfaire une specification donnee. Cette speci¬ 
fication peut comprendre des criteres de securite, des mesures de rendement ou d’autres 
conditions. Le processus de verification, dont le but est de verifier que le systeme sat- 
isfait une specification, peut etre realise de differentes manieres. Chacune ayant ses 
avantages et inconvenients, le choix de la methode a utiliser depend a la fois du sys¬ 
teme qui est etudie et de la specification. Une des methodes de verification possibles 
est de realiser des batteries de tests sur le systeme. Cette methode est notamment 
utile lorsqu’on ne connait pas le fonctionnement interne d’un systeme (le code d’un pro¬ 
gramme par exemple). Le choix des tests est alors base sur la specification iGTW.KKfl . 
Au contraire, si l’on a acces au systeme complet, un modele formel et operatiounel du 
systeme peut etre construit. Ce modele peut ensuite etre etudie avec des methodes 
dediees. 

La complexite de certains systemes peut rendre la construction du modele difficile. 
Celle-ci peut etre accompli grace a une analyse du code du systeme, en realisant des 
tests specihques dans certaines configurations du systeme permettant de determiner 
comment il evolue (par exemple en surchargeant le processeur d’un ordinateur affii 
d’observer comment le programme reagit face a ce type de pression), etc. Quand elle 
est possible, cette approche a de nombreux avantages : 

• Lors de la conception d’un systeme, si le prototype actuel n’accompli pas les 
objectifs voulus, il doit etre modihe. Construire de nouveaux prototypes jusqu’a 
en obtenir un qui soit satisfaisant est couteux. Il est plus economique et simple, de 
modiher un modele jusqu’a ce que celui-ci satisfasse la specification, et seulement 
alors de construire le systeme associe. 

• Un modele est souvent congu ahn de verifier plusieurs proprietes. Si l’on desire 
verifier de nouvelles proprietes a une date ulterieure, utiliser le modele existant 
peut suffire. Dans le cas contraire, il n’est pas forcement necessaire de construire 
entierement un nouveau modele. Il peut etre suffisant de raffiner le modele actuel, 
en y incorporant les informations appropriees. Utiliser un tel raffinement reduit 
fortement la complexite de la nouvelle etude. 

• Enfin, si le modele est suffisamment proche de la realite, il permet une analyse 
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precise du systeme. Ce n’est pas le cas lors d’utilisation de batteries de tests, qui 
ne couvrent qu’une partie des situations possibles. Le meme souci existe pour 
d’autres methodes telles que la verification statistique de modele [Bar ni- 

ii existe de nombreux formalismes permettant de representer un systeme. Plus celui- 
ci est complexe (representation du temps, de l’aleatoire...), plus on peut modeliser de 
systemes et de specifications et plus l’etude de ce fornralisme est difficile. Notam- 
ment, les formalismes possedant une composante probabiliste tels que les cfiaines de 
Markov |KS60j ont de nombreuses applications. En effet, certains systemes necessitent 
des probabilites pour etre represented de fagon precise. C’est le cas des systemes con- 
tenant des comportements intrinsequement probabilistes, par exemple un programme 
utilisant l’aleatoire afin de briser les symetries. Le hasard peut aussi etre une con¬ 
sequence de l’interaction du systeme avec l’environnement dont le comportement n’est 
pas entierement previsible. De plus, les probabilites permettent de representer les incer¬ 
titudes d’un modele construit de fagon approchee, par analyse statistique par exemple. 
Enfin, l’utilisation de probabilites elargit l’ensemble des proprietes qui peuvent etre 
specifiees en permettant de les quantifier. Par exemple, si un systeme (qui n’est pas 
un systeme critique) peut commettre des erreurs, mais que celles-ci ont peu de chances 
d’avoir lieu, ceci peut suffire pour satisfaire la specification. 

Le choix du fornralisme utilise determine egalement quelles sont les informations 
accessibles aux utilisateurs : lorsque l’on construit un modele, les differents evenements 
qui peuvent avoir lieu dans le systeme et leurs effets sont decrits ; certains de ces 
evenements ont lieu de fagon interne et ne sont done pas observables par un utilisateur 
exterieur. Le controle de l’information transmise par un systeme a vu son importance 
augmenter ces dernieres annees a cause de l’omnipresence des instruments electroniques 
communicants. Certaines informations du systeme doivent etre maintenues secretes 
(les mots de passe par exemple) alors que d’autres doivent etre rendues publiques (les 
erreurs du systeme par exemple). Les problemes lies a Vobservation partielle peuvent 
etre groupes en trois families selon le type d’objectif a accomplir : (1) la planification 
sous observation partielle, (2) la dissimulation d’information a l’observateur et (3) la 
recuperation d’information. 

Le diagnostic est l’un des problemes principaux de cette troisieme categorie. Le 
terme diagnostic vient du domaine medical dans lequel il designe 1’identification d’une 
maladie a partir de symptomes. Dans la communaute des systemes a evenements dis- 
crets, cette identification est appliquee a des systemes dynamiques (les centrales elec- 
triques, les chaines de production.. .). Dans cette approche, une execution du systeme 
est observee et on essaie de detecter si un evenement particulier, appele la faute, a eu 
lieu. La faute ne represente pas forcement une defaillance du systeme, cependant cette 
terminologie est utilisee principalement car une irregularite est l’un des evenements les 
plus importants a detecter durant une execution. En effet, celles-ci menacent la surete 
et la disponibilite du systeme. Ceci peut provoquer des degats catastrophiques a la 
fois en termes economiques et humains. L’etude des fautes est egalement justifiee du 
fait que tout systeme peut, et en fait va, faire une erreur. En effet, les systemes que 
l’on construit sont de plus en plus complexes et ont des interactions de plus en plus 
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importantes avec l’environnement. II est done extremement difficile de ne pas intro- 
duire d’erreurs lors de la conception d’un systeme et il est presque impossible de predire 
toutes les actions que l’environnement aura sur le systeme. Enfin, des fautes auront lieu 
a cause du vieillissement des composants du systeme. 

Comrne les fautes sont dangereuses, inevitables et potentiellement difficiles a identi¬ 
fier, une methode automatique de detection est necessaire. Par ailleurs, cette methode 
doit etre precise car stopper un systeme a cause d’un faux positif est couteux et elle doit 
etre reactive de faqon a ce que les fautes soient reperees avant tout dommage serieux. 
En reaction a une faute, on peut soit (1) essayer d’optimiser le comportement du sys¬ 
teme durant la periode prealable a la faute |EMT16j . ce qui est particulierement utile 
pour des systemes dont les composants sont regulierement remplaces, ou (2) essayer de 
detecter la faute de faqon a reagir a son occurrence. Comme l’on desire reagir aussi 
vite que possible, predire l’occurrence de la faute permettrait de reagir avant meme 
que le systeme ne commette l’erreur. Cette question est etudiee dans le contexte des 
problemes de prediction |GL09| . Cependant, il est rare qu’un systeme permette une pre¬ 
diction efficace des fautes. Detecter les fautes a posteriori est plus plausible. L’etude 
du diagnostic souleve deux problemes inrportants : comment decider si un systeme est 
diagnostiquable, ce qui est appele diagnostiquabilite, et, dans le cas positif, comment 
construire un diagnostiqueur la fonction realisant le diagnostic et qui satisferait poten¬ 
tiellement des conditions supplementaires sur la taille de la memoire utilisee, le delai de 
detection, etc. Dans le domaine des systemes a evenements discrets, le diagnostic a ete 
deffih initialement pour des systemes finis tels que les systemes a transitions etiquetees 
partiellement observables |SSL + 95| puis a ete etendu a de nombreux modeles complexes 
( e.g. les reseaux de Petri |CGLS12jlBHSS18] . les systemes a pile [MP09| . etc.) et cadres 
( e.g . decentralises iPETOOj . distribues |HC94| b De plus, plusieurs travaux rassenrbles 
sous le nom diagnostic actif etudient comment controler le systeme pour en assurer la 
diagnostiquabilite |SLT981 ITT071ICT081 ICP09| . 

Notre but dans ce document est d’etudier le diagnostic de systemes probabilistes. 
Par consequent, la premiere question qui se doit d’etre abordee est le choix du formalisme 
(probabiliste) a utiliser. On doit notamment determiner si, en plus des probabilites, le 
modele est partiellement controlable, s’il peut representer une infinite d’etats differents 
ou s’il doit decrire efficacement des comportements concurrents. Par ailleurs, comme 
le diagnostic est un probleme d’observation partielle, le modele doit indiquer quelle 
observation est associee a une execution. 

Dans un deuxieme temps, il va nous falloir etablir les differents problemes que nous 
allons etudier. De nombreuses notions de diagnostic ont deja ete definies, chacune ac- 
complissant un objectif different. Nous devons done presenter un ensemble coherent 
de notions qualitatives et quantitatives appropriees englobant les definitions impor¬ 
tantes deja etablies. De plus, les definitions formelles des problemes que nous etudions 
doivent etre choisies prudemment. En effet, comme ces problemes melangent observa¬ 
tion partielle, probabilites et, dans certains cas, controle, ils ont de fortes chances d’etre 
indecidables. Une petite modification dans la definition peut faire la difference entre un 
probleme pouvant se resoudre efficacement et un probleme indecidable. 

Une fois les notions de diagnostic definies, notre but sera d’etablir les complexites 
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precises de la diagnostiquabilite et de la synthese des diagnostiqueurs pour chaque no¬ 
tion, ceci, dans les differents formalismes que nous aurons choisis. Nous chercherons 
egalement a determiner comment modifier un systeme afin qu’il devienne diagnosti- 
quable. Ceci donne deux approches: une approche passive, de verification, et une 
active, de controle. 


Definitions du modele et du diagnostic 


Le modele le plus utilise dans ce document est celui des systemes probabilistes a tran¬ 
sitions etiquetees partiellement observables (dont le nom anglais est abrege en pLTS). 
Un pLTS est formellement defini par un tuple {Q, qo, X, T, P) ou Q est un ensemble 
denombrable d’etats, qo est l’etat initial, X est un ensemble d’evenements pouvant avoir 
lieu dans le systeme, T est un ensemble de transitions indiquant comment un evenement 
de X affecte l’etat courant du systeme et P donne la probabilite de chaque transition. 
Afin de representer l’observation partielle du systeme, l’ensemble d’evenements X est 
partitionne entre evenements observables X„ et evenements inobservables T, u . 

Une execution p du systeme est une suite d’etats et de transitions liant deux etats 
consecutifs. Grace a P, on peut attribuer une probabilite a toute execution finie. En- 
suite, et en utilisant des resultats de theorie de la mesure, on peut definir une mesure 
de probabilite sur l’ensemble des executions infinies. Par ailleurs, a toute execution on 
peut associer une observation qui est la projection sur X G de la sequence d’evenements 
etiquetant ses transitions. On suppose que le pLTS est convergent ce qui signifie que 
toute execution infinie possede une observation infinie. 


Example 0.1. Considerons le pLTS represente dans la figure 0.1 Une utilisation 
normale de la machine a cafe est donnee par exemple par I’execution p = qo piece 
qi sucre q\ cafe qo. Cependant dans I’etat q\ une erreur representee par Vevenement 
‘f ’ peut avoir lieu, menant a I’etat f\ d partir duqiLel on ne peut plus obtenir de cafe. 
Cet evenement a cependant une faible probabilite d’avoir lieu. L’execidion normale 
p a pour probabilite le produit des probability des transitions empruntees, c’est-a-dire 
1 x 0.29 x 0.7 = 0.203. Un comportement fautif de la forme p' = qo piece gq sucre q± 
f /i a probabilite 0.0029 d’avoir lieu. 


Comme dans l’exemple precedent, les executions du modele peuvent etre fautives 
ou correctes. Ceci est indique par la presence (ou absence) de la faute (l’evenement 
f) a l’interieur de celle-ci. Le but du diagnostic est d’utiliser l’observation d’une exe¬ 
cution pour determiner si celle-ci est correcte ou fautive. Comme plusieurs executions 
differentes peuvent posseder la meme observation, une execution est surement fautive 
(resp. surement correcte) si toute les executions partageant la meme observation sont 
fautives (resp. correctes). Sinon, l’execution est ambigue. Si une execution est surement 
correcte ou surement fautive, un verdict peut etre rendu. Le souci vient des executions 
ambigues. 
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sucre, 0.29 hors service, 1 



Figure 0.1: Un pLTS representant une machine a cafe, c/o est l’etat initial, ce qui est 
represente par la fleche entrante. Les transitions entre les etats sont etiquetees par 
l’evenement provoquant cette transition ainsi que par la probabilite que cette transition 
soit prise. 


Reactivite 


Executions finies 


FF-diagnostic 


FA-diagnostic 


Executions infinies 


I F-diagnostic 


lA-diagnostic 


-*-*-► Verdict 

Executions fautives Toutes les executions 

Figure 0.2: Resume des variantes du diagnostic exact. 


Plusieurs definitions de diagnostiquabilite peuvent etre proposees. Pour des sys- 
temes non probabilistes, la definition originelle de diagnostiquabilite requiert que toute 
sequence fautive devienne finalement surement fautive |SSL + 95] , Ainsi, toute faute 
est finalement detectee. Cette condition est trop forte pour des systemes probabilistes 
car un systeme pourrait etre declare non diagnostiquable a cause d’une execution de 
probabilite nulle. Une adaptation possible est de demander qu’avec probabilite 1 une 
execution fautive devienne surement fautive |TT05| . Nous appelons cette notion FF- 
diagnotiquabilite. Cette notion ignore l’ambigui'te des executions correctes. Le systeme 
peut done rester ambigii infiniment, ce que l’on peut vouloir eviter. La diagnostiqua¬ 
bilite peut etre etendue aux executions correctes en requerant que la probabilite des 
sequences ambigiies (fautives et correctes) converge vers 0. Cette notion se nomme FA- 
diagnostiquabilite (le A signifie "all" alors que F signifie fautif). Pour ces deux notions de 
diagnostiquabilite, l’ambigu'ite est resolue sur des executions finies (le premier F signifi- 
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ant justement fini). Si l’on autorise l’ambiguite a etre resolue lorsque la sequence devient 
infinie, la FF- et la FA-diagnotiquabilite deviennent la IF- et la lA-diagnotiquabilite. Les 


quatre notions sont resumees dans la figure 0.2 


Ces notions de diagnostiquabilite ne permettent aucune erreur de verdict. Ce sont 
des notions dites exactes. II peut cependant etre interessant d’affaiblir cette condition. 
Considerons le pLTS de la figure |CK3| Toute execution fautive est ambigue. Cependant, 
de par le choix des probability, une execution fautive a plus de chances de produire un 
l b ’ qu’un ‘a’ et inversement pour les executions correctes. Par consequent, en comparant 
le nombre de ‘6’ et de ‘a’ dans l’observation, on peut deduire avec forte probability si 
l’execution est correcte ou fautive. Nous considerons done plusieurs notions de diagnos¬ 
tiquabilite dites approchees dans ce document (l’une d’entre elle ayant ete introduite 
dans [TT05j et pour laquelle seule une condition suffisante avait ete donnee). 



Figure 0.3: Quand un diagnostic approche est necessaire. 


La formalisation du modele et des notions de diagnostiquabilite est realisee dans 
le chapitre [2] Nous expliquons maintenant comment ces notions de diagnostiquabilite 
peuvent etre etudiees. 


Verification de la diagnostiquabilite 

L’etude d’un probleme commence par une analyse semantique (developpees dans le 
chapitre [3]) de fagon a bien le comprendre. Dans le cas de la diagnostiquabilite, cette 
analyse semantique prend tout d’abord la forme d’une etude des liens entre les differentes 
notions de diagnostiquabilite. Certains sont assez clairs. Par exemple, les notions de 
FA- et lA-diagnostiquabilite considerent les executions fautives ainsi que les executions 
correctes alors que FF- et IF-diagnostiquabilite ne considerent que les notions fautives. 
Done un systeme FA-diagnostiquable (resp. IA-) est FF-diagnostiquable (resp. IF). Sim- 
ilairement, observer des executions infinies donne plus d’informations que leurs prefixes 
finis, done les notions de diagnostiquabilite finies impliquent leur equivalent infini. De 
fagon interessante, si on ne s’interesse qu’aux executions fautives, on a une reciproque 
partielle : si le systeme est a branchement fini, la FF-diagnostiquabilite est equivalente 
a la IF-diagnostiquabilite. 

La deuxieme etape d’une analyse semantique est de determiner des caracterisations 
efficaces des notions etudiees. Avoir des contraintes sur le systeme etudie permet d’avoir 
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des caracterisations plus simple. Nous etudions done d’abord le cas des systemes ayant 
un nombre d’etats fini. 


Etude des systemes finis 

Les systemes represente par un modele ayant un nombre d’etats fini possedent des pro- 
prietes extremement utiles pour le diagnostic. Notamment, on sait qu’avec probabilite 1, 
une execution atteint une composante strictement connexe terminate (CSCT) du graphe 
induit par le pLTS. Comme la diagnostiquabilite s’interesse a des comportements "avec 
probabilite 1", l’etude peut se concentrer sur les CSCT du systeme. Par ailleurs, il 
existe une methode simple de determinisation de l’automate induit par le pLTS. Cette 
determinisation est tres utile pour caracteriser l’ambiguite d’une execution. En effet, 
l’automate determinise associe a chaque sequence d’observations l’ensemble d’etats du 
pLTS pouvant etre atteint par des executions associees a cette sequence. Par consequent, 
en supposant sans perte de generality que les etats du pLTS sont partitionnes entre etats 
fautifs (atteint par une execution fautive) et etats corrects, une execution surement fau- 
tive est une execution dont la sequence d’observation mene a un etat de l’automate 
determinise ne possedant que des etats fautifs. Observons ceci sur un exemple. La 
figure [f)~T| represente un pLTS qui est FF-diagnostiquable ainsi que lA-diagnostiquable 
mais qui n’est pas FA-diagnostiquable. En effet, toute faute est suivie finalement par 
un ‘6’ revelant la faute. L’observation a u est done au contraire associee a une execution 
surement correcte, mais tout prefixe fini de cette execution est ambigu. Done toute 
execution correcte finie est ambigue. 



Figure 0.4: Un pLTS qui est IA et FF-diagnostiquable mais n’est pas FA-diagnostiquable. 


Le pLTS de la figure 0.4 respecte la partition entre etats fautifs (les /,;) et etats 
corrects (les qi) mentionnee plus tot. Nous representons l’automate determinise induit 
en figure [()~5j Un etat de cet automate atteint par l’observation w contient deux ensem¬ 
bles : nous separons les etats du pLTS pouvant etre atteint par une execution correcte 
de ceux atteints par une execution fautive, de plus on ne considere que les executions 
terminant par un evenement observable. Les etats doublement entoures ne contiennent 
soit aucun etat correct, soit aucun etat fautif. Les observations menant a ces etats 
correspondent done a des executions non ambigues. 

En observant cet automate et au vu de notre remarque anterieure sur les CFCT, on 
pourrait penser que le diagnostic de ce systeme est simple etant donne que le seul etat 
pour lequel le verdict ne peut etre rendu ne fait pas parti d’une CFCT de l’automate 
determinise. Cependant, si les CFCT sont atteintes avec probabilite 1 dans le pLTS, ce 
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Figure 0.5: L’automate determinise associe au pLTS de la figure 



n’est pas forcement le cas dans l’automate determinise. Afin de regagner cette propriete 
tout en conservant les informations donnees par l’automate determinise, on realise le 
produit synchronise sur les evenements observables du pLTS et de l’automate. Celui-ci 
est represente dans la figure |0.6[ 






a 9i, si 



Figure 0.6: Produit synchronise du pLTS de la figure [(h4| et de son automate determinise. 
Les probabilities sont omises pour faciliter la lisibilite. 


Le produit synchronise conserve le comportement probabiliste du systeme. C’est-a- 
dire, qu’il y a une bijection entre les executions du pLTS et celles du produit, les deux 
executions ayant la meme observation, correction et probability Par consequent, la 
diagnostiquabilite du pLTS est equivalente a celle de son produit synchronise. Par con- 
tre, ce dernier possede plus d’informations car la composante des etats correspondants 
a l’automate determinise indique si une execution finissant dans cet etat est surement 
fautive, surement correcte ou ambigiie. Observons les CFCT de ce systeme. II y en a 3, 
toutes reduites a un etat: (qi, s i), (/2, S2) et (/2, 54). Les ensembles S2 et 54 impliquent 
que les executions atteignant la deuxieme et troisieme CFCT sont surement fautives. 
si en revanche, montre une ambigui'te dans la premiere CFCT. Comme si est associe 
a un etat correct q \, cette ambigui'te n’existe que pour des executions correctes. Ce 
produit synchronise montre done que le pLTS est bien FF-diagnostiquable, mais pas 
FA-diagnostiquable. 
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En utilisant le produit synchronise, nne caracterisation peut etre etablie pour toutes 
les notions de diagnostiquabilite exact^] Cette caracterisation peut ensuite etre veri- 
fiee en espace polynomial. La principale source de complexity de l’algorithme est la 
determinisation qui produit un automate de taille au plus exponentielle en la taille du 
pLTS. Utilisant une reduction du probleme de l’universalite du langage genere par un 
automate non deterministe, on montre que les notions de diagnostiquabilite exacte sont 
PSPACE- difficiles. Elies sont done PSPACE-completes pour les pLTS finis. 

En ce qui concerne le diagnostic approche, seule une notion, construite en modifiant 
subtilement celle introduite dans |TT05j . est decidable. La decidability est montree en 
reduisant le probleme a un nombre au plus quadratique d’instances du probleme de 
la distance 1 de deux chaines de Markov etiquetees, probleme qui a ete montre decid¬ 
able en PTIME dans |CK14j . Ceci donne in fine un algorithme PTIME. Les resultats 
d’indecidabilite quant a eux sont montres grace a des reductions du probleme du vide 
des automates probabilistes [Paz nj. 

Construction des diagnostiqueurs 

Le but de l’etude du diagnostic est la detection automatique de la faute. Cette detection 
est realisee par un diagnostiqueur qui observe le systeme et donne son verdict. Formelle- 
ment, un diagnostiqueur est une fonction Z) : E* —»{?,T,_L}. Un verdict ? ne four nit 
aucune information, un verdict T declare 1’execution actuelle fautive et un verdict _L 
fournit une information relative a la correction de l’execution. Un diagnostiqueur a trois 
caracteristiques principales: verdict, surete et reactivite. Le verdict formule la nature 
de l’information que le diagnostiqueur doit fournir au cours d’une execution (detection 
de fautes uniquement, ou aussi de la correction de l’execution par exemple). La surete 
formule quand le diagnostiqueur peut emettre son verdict. Dans le cas du diagnostic 
exact, la surete requiert que si le diagnostiqueur produit un verdict, celui-ci est correct. 
Ce n’est pas forcement le cas dans le cadre du diagnostic approche. La reactivite ex¬ 
prime a quelle regularity le diagnostiqueur doit fournir des informations sur le statut de 
l’execution courante. 

Les notions de diagnostiquabilite ayant ete presentees sous la forme d’un probleme de 
decision, la troisieme etape de l’etude semantique consiste a etablir le lien entre chaque 
notion de diagnostiquabilite et l’existence d’un diagnostiqueur avec un verdict, une 
surete et une reactive donnes. La preuve permettant d’etablir le lien entre diagnostiqua- 
bilite et existence d’un diagnostiqueur est constructive. Par consequent, nous disposons 
d’un algorithme permettant de construire automatiquement un diagnostiqueur pour 
chaque systeme diagnostiquable. Cependant, le diagnostiqueur en question utilise une 
memoire non bornee. En vue d’une possible implementation, il est preferable de pouvoir 
se limiter a des diagnostiqueurs a memoire finie. Un tel diagnostiqueur est represente 
par un automate deterministe sur E 0 enrichi d’un verdict (M, E 0 , mo, up, Df m ) ou M 
est un ensemble d’etats de la memoire avec uiq l’etat initial, up est la fonction de tran- 

1 Pour la [A-diagnostiquabilite, une information supplementaire est necessaire. L’automate deter- 
minise que l’on construit possede un troisieme ensemble permettant de partitionner les etats fautifs en 
deux groupes, selon le moment ou la faute a ete commise. 
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sition, mettant a jour la memoire du diagnostiqueur et Dj m : M —>• {?, T, _L} associe 
un verdict a chaque etat de la memoire. La taille d’un tel diagnostiqueur a memoire 
finie est le nombre d’etats de la memoire qu’il possede. La fonction de mise a jour peut 
etre etendue a des sequences d’observations de fagon inductive : pour e le mot vide 
w 6 E* et a G E 0 , up(m, e) = met up (m,wa) = up(up (m,w),a). Si un diagnostiqueur 
a memoire finie n’est pas un diagnostiqueur selon la definition etablie plus haut, il en 
induit un qui est definit par D(w ) = Df m (up(m.Q, w)) pour tout w E £*. 

Example 0.2. La figure \0. 7| represente un diagnostiqueur a memoire finie qui ne fournit 
aucune information (verdict ?) initialement puis declare une faute (verdict T ) des qu’un 
‘b ’ est observe. 

Considerons le pLTS de la figure \0.J\ La faute est identifiee des qiL’im ‘b’ est observe. 
Par consequent, le diagnostiqiLeur induit par le diagnostiqueur a memoire finie de la 
figure \ 0. 7| peut etre utilise pour detecter les fautes de ce pLTS. Ce diagnostiqueur ne 
detecte pas les executions correctes, il correspond a la notion de FF -diagnostiquabilite. 
On I’appelle done un FF-diagnostiqueur. 

a a,b 

b 


? T 

Figure 0.7: Exemple de diagnostiqueur a memoire finie. Le verdict donne par Dj m dans 
un etat de la memoire est indique sous l’etat. 




Pour toute notion de diagnostic exact, un diagnostiqueur peut etre construit a partir 
de 1’automate determinise qui a ete utilise pour verifier la diagnostiquabilite. Le diag¬ 
nostiqueur peut done avoir une taille exponentielle. C’est malheureusement inevitable : 
certains pLTS n’admettent pas de diagnostiqueurs de taille sous-exponentielle. Pour 
le diagnostic approche, aucune borne sur la taille de la memoire n’existe dans le cas 
general. En d’autres mots, on ne peut pas toujours construire de diagnostiqueur a me¬ 
moire finie. Pire encore, determiner s’il existe un diagnostiqueur a memoire finie est un 
probleme indecidable. 

Ces resultats, reposant fortement sur la limitation a un nombre d’etats finis pour le 
modele, sont rassembles dans le chapitre|4] Nous allons maintenant discuter de ce qui 
peut etre fait pour lever cette restriction. 

Etude des systemes infinis 

De nombreux systemes reels necessitent un nombre infini d’etats pour etre decrit de 
faqon precise. Afin de rendre une analyse possible, on ne peut pas utiliser directement 
un pLTS infini. On a besoin d’un modele de plus haut niveau, capable de representer de 
fagon finie un pLTS infini. De nombreux formalismes permettent ceci. Les automates a 



15 


pile et les reseaux de Petri notamment representent deux classes orthogonales de LTS 
infinis. Le choix du formalisme est tres important car plus un formalisme est expressif, 
plus les problemes seront compliques a resoudre. Les automates a pile et les reseaux de 
Petri sont trop puissants par exemple, toutes les notions de diagnostiquabilite exacte 
etant indecidables dans ces deux formalismes. 

Nous nous sommes done interesses a un formalisme legerement plus faible, plus 
precisement une sous-classe des automates a pile : les automates a pile visibles. Les 
automates a pile sont des automates enrichis d’une pile qui leur permet de conserver 
de l’information au corns d’une execution. Les transitions de l’automate disponibles 
dependent de l’etat courant ainsi que de la tete de la pile. Une fois selectionnee, une 
transition peut soit (1) modifier la tete de la pile (transition locale), (2) ajouter un 
nouveau symbole en tete de pile (transition d’empilement) ou (3) retirer la tete de pile 
actuelle, s’il y en a une (transition de depilage). La semantique d’un automate a pile 
probabiliste est un pLTS infini dont les etats representent l’etat actuel de l’automate 
a pile ainsi que le contenu de la pile. Ce pLTS peut etre infini car la taille de la pile 
n’est pas bornee. La restriction aux automates a pile visible requiert que l’ensemble 
des evenements est partitionne selon le type de transitions auquel ils correspondent, 
T, = Sj( U £|, U S[|. Les evenements de Sj, £, et sont respectivement associes a des 
transitions d’empilement, de depilage et des transitions locales. De plus, Sj U C S 0 . 
Un observateur voit done quand un element est ajoute ou retire de la pile. La taille de 
la pile est done connue a tout moment, son contenu par contre peut etre inconnu. 


Example 0.3. La figure 0.8 donne un exemple d’automate a pile probabiliste. Une 
execution demarre dans I’etat qo avec pour contenu de pile le symbole J_o- Celui-ci est 
appele element de fond de pile et ne peut pas etre retire ou modifie. Le reste de la pile 
n’est compose que d’un certain nombre de 7 . La seide transition d’empilement est in et 
les transitions de depilage sont out et abort. 

Ce systeme regoit un certain nombre d’ordres qu’il note dans la pile. Puis il com¬ 
mence a servir ses clients, chaque ordre regu regoit done une reponse soit sous la forme 
d’un out, soit de fagon fautive sous la forme d’un abort. Enfin, il retourne a son point 
initial. Les deux executions donnees en exemple presentent un comportement correct et 
fautif pour deux ordres regus. 

Il y a une claire partition entre transitions d’empilement, de depilage et locales. Par 
consequent si in, out et abort sont observables, cet automate a pile probabiliste est visible. 


La restriction aux automates a pile probabilistes visibles limite peu l’expressivite du 
formalisme, mais donne des proprietes supplementaires utiles au modele, notamment 
elle permet de realiser une determinisation de l’automate a pile. On ne peut cependant 
pas faire comme dans les systemes finis et etudier les CFCT du produit du modele et 
de son automate determinise. Notamment car ces CFCT peuvent ne pas exister (cas ou 
aucune borne sur la taille de la pile n’existe). Une autre forme de caracterisation de la 
diagnostiquabilite est done necessaire. Pour ce faire, une logique nominee pathL a ete in- 
troduite et des formules caracterisant plusieurs notions de diagnostiquabilite exacte ont 
ete etabli. Ensuite, en utilisant notamment l’automate a pile determinise, ces formules 
ont pu etre traduites en des formules de pLTL, une logique connue et pour laquelle des 
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Figure 0.8: Un automate a pile probabiliste et deux de ses executions finies. 


algorithmes de verification existe pour les automates a pile. En utilisant ces resultats 
nous avons obtenu des algorithmes EXPSPACE pour les notions de diagnostiquabilite 
caracterisees (la borne inferieure prouvee etant EXPTIME). La FA-diagnostiquabilite, 
qui requiert la detection des sequences fautives et correctes en temps fini n’a cependant 
pas pu etre caracterisee. En fait, des resultats de non-expressivite ont ete etabli pour 
montrer que cette notion ne pouvait pas etre exprimee en pathL. 

Ces resultats sont developpes dans le chapitre[5] 


Controle d’un systeme 


Les pLTS donnent une representation passive d’un systeme. Ainsi, etudier le diagnostic 
sur des pLTS est purement un travail de verification. II ne permet done pas de ques- 
tionner efficacement comment modifier le systeme afin de le rendre diagnostiquable. 
Afin d’etudier ce genre de probleme, on ajoute une forme de controle dans le pLTS. Le 
formalisme obtenu, appele CLTS partitionne l’ensemble des evenements observables en 
evenements controlables E c et evenements incontrolables X e . Apres chaque observation, 
un controleur choisit un ensemble d’evenements autorises excluant potentiellement cer¬ 
tains evenements controlables. Ceci limite done les transitions pouvant etre prise par le 
systeme. 


Example 0.4. Un exemple de CLTS est represente dans la figure 0.9. Initialement, 
deux evenements de poids 1 sont possibles et forcement autorises par le controle. Par 
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a, 1 a, 1 



a, 1 b, 1 


Figure 0.9: Un exemple de CLTS. Les probabilities sont remplacees par des poids. Le 
seul evenement controlable est l b\ 


consequent chacune a une probability \ d’etre prise. En q\, ‘b’ peut etre interdit par le 
controleur. Si c’est le cas, la transition etiquetee par ‘a’ est choisie avec probability 1. 

Ainsi, si le controleur autorise tous les evenements en permanence, I’execution 
qoUQiaQibq 2 a probability 1/8. S’il interdit ‘b’ en permanence, cette execution a proba¬ 
bility 0. Finalement, s’il n’autorise un ‘b’ qu’apres I’observation d’un ‘a’, I’execution a 
probability 1/4. 


Le controle est formellement defini par l’utilisation de strategies. Une strategie n : 
X* i—^ Dist(2 s ) est une fonction associant a une sequence d’observations une distribution 
probabiliste sur les ensembles d’evenements autorises. Par ailleurs, si l’ensemble X* est 
selectionne par la strategie, on a T, u U X e C X*. En d’autres mots, la strategie ne peut 
exclure que des evenements controlables. Un CLTS C equipe d’une strategie n genere 
un pLTS infini denote C n . 


Example 0.5. Considerons le CLTS C represente dans la figure |?77i[ II y a deux en¬ 
sembles d’evenements possibles a autoriser: X et X \ {6} que nous abregeons en X~. 
Definissons la strategie ir par n (a n ) = p n ■ X - + r n ■ X avec p n + r n = 1 pour tout n £ N 
etTi{w) = 1-X sinon. C’est-a-dire, apres avoir observe a n , avec probability p n I’ensemble 
X - est autorise par la strategie et avec la probability complementaire X est autorise. Le 
pLTS genere C n est infini. Une partie de celui-ci est represente en figure 0.10 


Expliquons la distribution de probability a la sortie de la configuration (e,(/i,X). 
Les deux transitions sortant de q\ ont le meme poids, comme elles sont toutes deux 
autorisees par la strategie, la probability de chaque transition est /. Comme ‘a’ et 
‘b’ sont observables, un nouveau controle est choisi. Si un ‘b’ est observe, de par la 
definition de n, le nouveau controle est X. Par contre, si un ‘a’ est observe, le nouveau 
controle est X~ avec probability p\ et X sinon. II y a done trois transitions sortantes de 
(e,gi,X) ay ant pour probability respectivement 0.5, 0.5pi et0.5r\. 


Nous discutons maintenant des questions abordees pour les CLTS. 
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Diagnostic actif et degradation 

Lorsque l’on etudie le diagnostic d’un CLTS, la question n’est plus si le CLTS est diag- 
nostiquable (ce qui n’a techniquement pas de sens en soi) mais s’il existe une strategic 
telle que le pLTS induit est diagnostiquable. Cette problematique a ete etudiee pour 
des systemes probabilistes dans [BFH^14j. Afin de determiner s’il existe une strategie 
satisfaisant la notion de diagnostiquabilite qu’ils etudient, ils traduisent le probleme du 
diagnostic en une condition de Buclii pour un processus de decision Markovien partielle- 
ment observable. Le probleme peut ensuite etre resolu avec des techniques connues. La 
strategie obtenue a de bonnes proprietes : elle est notamment ce qu’on appelle "basee 
sur la croyance", ce qui implique entre autres que le pLTS genere est fini. 

Leur travail souleve un souci important : afin de rendre le systeme diagnostiquable, 
la strategie peut faire des choix problematiques comme forcer l’occurrence d’une faute. 
Bien que ceci permette de detecter le mauvais comportements du systeme, cela va a 
l’encontre du but initial du diagnostic qui est de pouvoir utiliser un systeme fonctionnel. 
Ils introduisent done le probleme du diagnostic sur. Celui-ci demande s’il existe une 
strategie satisfaisant a la fois le diagnostic et assurant une probability positive aux 
executions correctes. Cette notion est mallieureusement indecidable dans le cas general 
et un algorithme NEXPTIME est donne dans le cadre limite des strategies a memoire 
finie {i.e. pour lesquelles le pLTS engendre est fini). 

Continuant sur cette idee, nous avons introduit de nouvelles notions permettant 
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de mesurer la degradation du systeme. Ces notions ne s’assurent pas que le systeme 
reste correct infiniment avec une probability positive comme le diagnostic sur, rnais 
demandent que, si faute il y a, celle-ci puisse etre retardee fortement. Le diagnostic 
longtemps correct et le diagnostic fortement resistant sont deux telles notions, mesurant 
differemment le delai a imposer a l’occurrence de la faute. Ces deux notions sont 
impliquees par le diagnostic sur et sont differentes quand appliquees a des pLTS infinies 
{i.e. il existe des pLTS infinis satisfaisant chacune des notions sans satisfaire l’autre). 

Parmi les notions de degradation que nous avons introduites, certaines sont quan- 
titatives, d’autres, comme le diagnostic longtemps correct et le diagnostic fortement 
resistant, sont qualitatives. Les notions quantitatives sont toutes indecidables, meme 
limitees a des strategies a memoire finie. Au contraire, des algorithmes ont pu etre 
etablis pour les notions qualitatives. Ceux-ci precedent en deux etapes. Tout d’abord 
et en enrichissant les etats du CLTS comme fait dans le cas passif grace a une deter- 
minisation du CLTS, on identifie l’ensemble des etats du CLTS enrichi que l’on peut 
visiter tout en respectant la diagnostiquabilite. Cette methode permet en fait de con- 
struire la strategie la plus permissive assurant la diagnostiquabilite du systeme. Dans 
un second temps, on etudie le CLTS reduit aux etats accessibles sous cette strategie 
et on identifie comment restreindre la strategie afin de rester suffisamment longtemps 
dans une execution correcte. Cette etude peut se realiser en EXPTIME. Le diagnostic 
actif etant EXPTIME-difficile, les notions qualitative de degradation introduites sont 
done EXPTIME-completes dans le cas general. Nous avons egalement montre comment 
reduire a EXPTIME la complexity du diagnostic sur limite aux strategies a memoire 
finie. Sous cette restriction, le diagnostic sur est done EXPTIME-complet egalement. 

Ces travaux sont presentes dans le chapitre [6] 

Assurer l’opacite d’un systeme 

Dans cette these, le diagnostic est le probleme d’observation partielle auquel nous avons 
prete le plus d’attention. D’autres problemes d’observation partielle sont egalement 
interessants a etudier, notamment l’opacite que nous etudions formellement dans le 
chapitre [7J Le but de l’opacite est de cacher une information a l’observateur. En 
consequent, sur bien des aspects cette notion apparait comme un dual du diagnostic. 

Formellement, un systeme possede deux types d’executions : publiques ou secretes. 
Pour determiner, si une execution est secrete, on pourrait faire comme pour le diagnostic 
et utiliser un evenement particulier qui, s’il est present dans une execution, la rend 
secrete. De faqon equivalente, ceci peut etre represente en partitionnant l’ensemble 
des etats du systeme en etats publics et etats secrets et en considerant ces derniers 
absorbants. Une execution est secrete ici si elle visite un etat secret. C’est cette seconde 
option que nous utilisons pour l’opacite. Parmi les executions secretes, certaines revelent 
le secret. Ce sont celles pour lesquelles toute execution ayant la meme observation est 
secrete. Notons le parallele avec le diagnostic : en considerant les executions secretes 
comme fautives, une telle execution serait appelee surement fautive. Lorsqu’on etudie 
l’opacite d’un systeme, nous desirons mesurer a quel point le secret est revele, e’est-a- 
dire quelle est la mesure de probabilite des executions revelant le secret. Cette mesure 
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est appelee la revelation. 

L’opacite a ete etudiee dans un contexte passif, ainsi que dans un contexte actif. 
Le controle utilise dans l’etude de l’opacite est fortement different de celui utilise pour 
le diagnostic. En effet, le controleur est une fonction associant a une execution (et 
non a une sequence d’observations) une distribution sur un ensemble d’action. Chaque 
action correspond a une distribution de probability sur un ensemble de transitions. 
Ce formalisme donne beaucoup plus de puissance au controleur. II connait precisement 
quelle est l’execution actuelle, et son choix n’est pas forcement limite a quels evenements 
controlables il autorise, mais a un ensemble d’actions decrivant potentiellement des choix 
plus complexes. 

Autre difference avec les CLTS, dans le cadre de l’opacite, les observations ne sont 
plus mises sur les transitions, mais sur les etats. Pour noter ces differences, on parlera 
d’OMC pour les systemes passifs et d’OMDP pour les systemes controlables. 


Example 0.6. Considerons I’OMC represente en figure 0.11 L’observation associee 


avec chaque etat est indiquee a cote de celui-ci. Les etats secrets sont indiques en grise. 
En supposant que oi et 02 sont deux observations autre que e, toute execution contenant 
au moins 3 observations revele le secret. La revelation est done de 1. 


o 2 o 2 



Considerons maintenant I’OMDP de la figure \ 0.12 Dans I’etat initial go, deux 


actions sont possibles. Si l’action ‘a’ est choisie, l’execution entre en q\ avec probability 
2 et en q 2 avec la meme probability. Si ‘b ’ est choisie, tous les etats ont une probability 
| d’etre atteint. 

Definissons la strategic ir choisissant initialement Vaction ‘b’, puis toujours Vaction 
‘a ’. L ’OMC induit par ce OMDP controle par la strategic ir est celui represente dans la 
Ainsi, en utilisant la strategic 7 r, on assure une revelation de 1. En utilisant 


figure 0.11 


une strategic tt' qui selectionne ‘b’ a tout les coups, la revelation n’aurait ete que de \. 
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Figure 0.12: Un example d’OMDP. Les transitions sont etiquetees par des paires d’action 
et de la probabilite de prendre cette transition si cette action est choisie. 


Intuitivement, lorsqu’on controle un systeme pour augmenter la revelation, on realise 
une analyse du pire cas pour le systeme. Dans la realite, ce pire cas est atteint si, par 
exemple, le controle est effectue par un virus ayant accapare certaines fonctionnalites 
du systeme. L’inverse, quand le controle cherche a minimiser la revelation, est aussi 
interessant a etudier. Cela represente, par exemple, le cas ou un concepteur possede 
quelques degres de liberte dans son systeme et desire choisir 1’option qui maximiser a 
l’opacite du systeme. Ces deux problemes semblent symetriques au premier abord, mais 
leur analyse est en fait extremement differente. 

Pour la maximisation de la revelation, une simplification est possible au niveau des 
strategies : les strategies deterministes sont sufRsantes. C’est-a-dire, afin de maximiser 
la revelation, on peut se contenter de considerer des strategies qui associent a chaque 
execution finie non pas une distribution sur les actions, mais directement une action. 
Malgre cette simplification cependant, presque tous les problemes sont indecidables. La 
seule question importante que l’on peut resoudre est : etant donne une OMDP M, ex- 
iste t-il une strategie n telle que l’OMC induite a une revelation de 1. Ce probleme 
peut se traduire en un probleme d’accessibilite avec probabilite 1 dans un processus 
de decision Markovien partiellement observable, probleme pour lequel des algorithmes 
efficaces existent. Le processus construit est de taille exponentielle et l’algorithme re- 
solvant le probleme d’accessibilite est en EXPTIME. Par consequent une application di- 
recte donne un algorithme 2EXPTIME. Cependant, une analyse precise montre qu’une 
seule exponentielle est necessaire. En effet, la complexity de l’algorithme vient prin- 
cipalement de l’utilisation d’une forme de determinisation du processus de decision 
Markovien partiellement observable. Hors, celle-ci est deja necessaire a la transforma¬ 
tion de l’OMDP vers le processus et n’a done pas a etre repetee. Le probleme est 
egalement EXPTIME-difficile (reduction depuis les jeux de securite a information par- 
tielle), il est done EXPTIME-complet. 

Pour la minimisation, la situation est differente, d’une faqon surprenante. Tout 
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d’abord, on ne peut pas se limiter a des strategies deterministes. La capacite du con- 
troleur a agir de faqon randomisee est importante pour rendre le systeme opaque. Pour 
autant, les problemes sont beaucoup plus facile a resoudre : la revelation exacte de 
l’OMDP peut etre calculee. En effet, bien que l’on ne puisse pas utiliser de strategies 
deterministes, on peut se limiter a un type de strategies particulier, que nous avons 
nomine quasi-deterministe. Ces strategies choisissent une action et lui associent une 
probabilite proche de 1, puis partagent le reste de la probabilite sur toutes les autres 
actions. N’utiliser que des strategies de cette forme permet de reduire le probleme de 
la minimisation de la revelation a un probleme d’accessibilite dans les processus de de¬ 
cision Markovien. Contrairement au cas de la maximisation, ceux-ci sont totalement 
observables et plus de problemes sont decidables dans ce cas. Notanunent, minimiser la 
probabilite d’accessibilite lorsque l’observation est complete peut etre realise en temps 
polynomial. Done conune le processus que l’on construit est de taille exponentielle, on 
obtient un algorithme EXPTIME. Le probleme n’est cependant pas prouve EXPTIME- 
difficile. La meilleure borne inferieure dont l’on dispose est PSPACE et est obtenue par 
reduction de la validite d’une formule booleenne quantifiee (QBF). 


Conclusion 

Cette these presente principalement une analyse des problemes lies au diagnostic de 
systemes probabilistes. Sa premiere contribution est de rassembler en un tout coherent 
les differentes definitions existantes sur ce probleme. Ceci permet a la fois de donner 
une base solide a la recherche presentee ici, et de servir de fondations a toute recherche 
future sur ce sujet. 

En deuxieme point, cette these explique comment verifier les notions de diagnosti- 
quabilite definies pour differents systemes, que ceux-ci soient constitues d’un nombre 
d’etats fini ou infini. Pour les systemes infinis, la decidability de certaines notions reste 
ouverte et certains algorithmes ne sont pas prouves optimaux. II reste done du travail 
a realiser dans cette direction. 

Le cas des systemes controlables a enfin ete etudie. Pour ceux-ci, un autre angle 
de questionnement a ete utilise : il ne s’agit plus seulement de determiner la diagnos- 
tiquabilite du systeme. En premier lieu, il s’est agi de combiner le diagnostic avec une 
limitation de la degradation du systeme. Combiner ces deux problemes ne fait sens 
que pour des systemes actifs : pour un systeme passif, les deux problemes peuvent etre 
verifies separement, au contraire, dans des systemes actifs, il peut exister pour chaque 
propriete une strategie la verifiant, mais aucune strategic ne satisfait les deux simul- 
tanement. Dans un second temps, il a ete question de l’analyse de l’opacite, une autre 
notion de controle de l’information produite par un systeme. 



Part I 

Introduction 
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Chapter 1 

General Introduction 


Model-based verification. Many critical systems must fulfil a given specification. 
This specification may include security criteria, efficiency measures or other kinds of 
requirements. The verification process, which checks if the system respects the specifi¬ 
cations, can be performed in several manners. Each one having its pros and cons, the 
choice of the best method to use strongly depends on the kind of systems to be verified 
and on the specification. One of the possible method of verification is to perform tests 
on the system. If one wants to test a program for example, yet does not have access to 
the internal structures or working of it, the tests can be realised through specification- 
based testing [GTWJ03]. In this method, one considers the program as a black-box and 
focuses on the specification in order to determine which inputs are the most likely to 
show a failure of the system. If one has access to the content of the program, one can 
build a formal and operational model of the system. Then this model can be analysed 
via dedicated methods. 

Building the model may be difficult for some complex systems. It can be done 
by analysing the code of the system, by making specific tests in its different states to 
understand its evolution (for example by overloading the CPU to see how a program 
reacts faced to this kind of stress), etc. When possible, this approach has many benefits: 

• When designing a system, if the current prototype does not satisfy our goals, it 
must be modified. Building iteratively a new prototype until one gets a good 
result is expensive. It is easier and cheaper to modify a model of the system until 
it satisfies the requirements and only then implement the system. 

• A model is often built for checking several properties. If one wishes to check for 
additional properties at a later date, verifying the existing model may be sufficient. 
If not, one does not necessarily have to build a fully new model. One only needs to 
refine the existing one with the appropriate, missing information, which strongly 
reduces the complexity. 

• Finally, if the model is close enough to reality, then it allows for an accurate 
analysis of the system. This is not the case when using arrays of tests which only 
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partially cover the range of possibilities. The same issue exists for other methods 
such as statistical model checking [Bar Hi- 


Probabilistic models. There exist several different formalisms for representing a 
system. The more complete the formalism is (by adding time, multiple players...), the 
more systems and specifications can be described in it, but also the more complex it is 
to study. 

In particular, stochastic models such as Markov chains [KS60| or Markov decision 
processes jPut94| have many applications. There are some systems that require proba¬ 
bilities in order to be accurately represented. For example, they can be used to represent 
systems that contains inherent random behaviours. This occurs in any program using 
randomisation in order to break symmetries for instance, such as the algorithms dealing 
with the consensus problem |AgulO| . The randomisation also appears in the processes 
used in the consensus problem as one might represent the possibility that these processes 
fail with some probability. Another example of application of stochastic models is the 
case of systems that face unpredictable behaviours from the environment. This can be 
the case for a server that receives requests. These requests have randomised content and 
their timing of arrivals can also be random. The latter requires to mix probabilities and 
time in the model, as in stochastic timed automaton [BBB + 14], Moreover, probabilities 
can also be used in the model to represent the uncertainty created when the modelling 
is done through a statistical analysis. 

Using probabilities also enlarges the set of properties that can be specified by giving 
a measure on the runs of the system. For example, if a non-critical system possesses 
failures, yet one can determine that they are not likely to occur, this may be enough. 
Let us also consider a security example: if an attacker tries a password and discovers 
that it is wrong, he technically gets an information, however this information is not 
important enough to be worrisome. With probabilities, the specification can quantify 
the properties of the system the designer wants to verify. Moreover, even a qualitative 
quantification is useful as it allows to neglect behaviours that are present in the model, 
yet have a zero probability of occurring. 


Paradigms of partial observation. Another important component of a system that 
can be modelled is related to the observation available to the users: when one builds 
a model, one describes the different actions that can be taken by the system, however, 
these actions may be internal and are not necessarily visible by an external observer. 
Managing the information exchanged with a system has shown increasing importance 
in recent years due to the omnipresence of communicating electronic devices. Some of 
the actions of the system may need to be kept private (passwords) while others must 
be made public (failures). The problems raised by partial observation can be grouped 
in three families thanks to the different types of goals the system has: (1) planning 
under partial observation, (2) hiding information from the observer and (3) getting 
information from the system. 

This first category appears for example when studying games. In a game of poker, 
a player has to select a decision based on some observations (his cards) and on some 
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partial information (his opponents choices), moreover probabilities are involved in order 
to determine the likelihood to draw a specific card. Such a case falls under the study of 
works such as |BGG09| where the authors look for almost-surely winning or positively 
winning strategies in stochastic games. Partially observable Markov decision process 
(POMDP) is the stochastic one-player (also called one and a half player) special case of 
the above (See (CDHlOj for algorithms to achieve qualitative objectives in POMDP). 
POMDP have been extensively used in the IA community, for example in order to plan 
the actions of a moving robot |KLC98j . 

Many works focus on hiding information from an attacker. For instance code ob¬ 
fuscation [BGI + 01 , albeit in a complete observation setting. In a partial observation 
setting, many theoretical hiding problems are gathered under the general name “ opac¬ 
ity ”. An opaque system hides an information by ensuring the observation given by 
a secret behaviour of the system will be identical to the observation triggered by a 
non-secret behaviour. A practical example is given in |ABCP13| where the authors 
investigate how to hide the position of a cellphone user (by randomising the position 
declared by the phone) while getting relevant answers to location-based requests. In 
a theoretical and stochastic setting, |BKM12l IBMS15) defined probabilistic measures 
of the opacity of a model. The studied model is passive: once defined, one cannot 
modify its behaviour in order to ensure better properties. A form of control is quickly 
introduced in |BMS15| and expanded in |BCS15l IBKMS161IBKMS18] . More precisely, 
the authors of (BCS15) investigate Markov decision processes (MDP) with or without 
partial observation and secrets given by the infinite language of an automaton (using 
various accepting conditions). In fBKMSlbl IBKMS18] , a model in between MDP and 
POMDP is used: the control is realised as in an MDP (thus the controller uses complete 
information) but the winning condition (opacity) uses partial observation and is thus 
more related to POMDP problems. One issue with these approaches is that they all 
rely on the black-box hypothesis: it is assumed that the opponent does not know how 
the non-determinism of the system is resolved. This hypothesis simplifies the problem, 
but is unrealistic in many cases. For example, the control within the system could be 
the result of a virus implanted by the attacker. In this case, it is natural for the attacker 
to know how the virus is implemented. 

On the opposite, if the goal is to get information from the system, the first question 
to answer is to determine the kind of information that must be detected. One possibility 
is to determine, given a set of partially observable systems, which system is producing 
the current observation. In [CK 1 lj . the authors investigate how far two labelled Markov 
chains are one to the other in terms of the probabilities of the observed behaviours. They 
use a distance to measure the importance of the difference between the two models and 
approximate (or in some cases compute) this distance. The identification of a system 
can be applied to other questions such as the identification of its initial state, the 
equivalence of Markov chains (when the distance is equal to 0, see |DHR08| ) or the 
monitoring of hidden Markov chains (HMC). In this last example, one monitor observes 
a random run of one of two given HMC and must determine which HMC is the origin 
with appropriately high probability. The case of the monitor required to be correct 
with probability 1 on infinite sequences was introduced in [SZFll| and solved using the 
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distance of |CK14| in |KS16| . Instead of identifying the current system, one can wish to 
obtain a specific information from the observations of the current system. We develop 
this kind of problems in the next part due to their importance within this thesis. 


Diagnosis. Diagnosis, from the greek “ dla'yvuxjlg 1 ' which means “to distinguish” or 
“to discern”, is by the definition of the wiktionary the “identification of the nature and 
cause of something (of any nature)”. This describes many different problems in various 
domains. In medicine, a doctor analyses the symptoms to deduce the illness causing 
them. There has been multiple works in order to automatise this kind of diagnosis 
such as the rule-based expert system MYCIN [ BS84| . Another approach in the medi¬ 
cal domain can be found in computer-aided diagnosis (PMK + 99] where the computer 
analyses medical image such as radios of a patient and points towards the abnormali¬ 
ties it detects. Due to the non-negligible number of false positives and negatives, these 
works are far from replacing the experts opinion. Forms of diagnosis can also be found 
in network management for example, where it is more often called fault management. 
More precisely, fault management has two aspects. The first one, passive, consists in 
receiving messages from the devices on the network and if an alarm was sent, to un¬ 
derstand the cause and react to it. The second one takes an active step by considering 
that a failing device may not be able to detect its own fault and warn the system. 
Thus, the fault manager will interact regularly to check the behaviour of the devices. 
Fault managing therefore requires to do tests, diagnosis and possibly reparation. These 
diagnosis notions deal with systems that can be extremely complex but that are most 
often static. One wants to identify the current status of the system from what it is 
emitting currently, the evolution of the system is not monitored. Diagnosis, as seen in 
the discrete event systems community, focus in contrast on dynamic systems. 


Diagnosis of discrete event systems. For many systems (power systems, manu¬ 
facturing systems...) one needs to take into account the evolution of the system when 
analysing it. Such systems can be analysed with the approach from the discrete event 
systems community. In this approach, while the system is running, one follows a run 
of the system and tries to deduce the occurrence (or absence of) of a specific event 
called the fault. While one may want to detect any kind of important action of the 
system, the term “fault” is chosen both to correspond to the name “diagnosis” which, 
as shown in the previous examples, is mostly used to detect failures, and because faults 
are often one of the most important elements to detect within a run. They threaten 
the safety and availability of the system. In many of the systems listed above, a safety 
issue may lead to catastrophic damages both in terms of economic and human loses. 
The study of faults in particular is also justified by the fact that every system may, and 
will, fail. Indeed, the systems we build are increasingly complex and have increasingly 
intricate interactions with the environment. It is thus extremely difficult when designing 
a system not to introduce errors and it is almost impossible to predict every reaction 
the environment will have to the system. Finally, at the very least, failures will occur 
because of components ageing. 
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As faults are dangerous, unavoidable and potentially hard to detect (especially in 
large-scale complex systems), one needs an automated way to detect them. Moreover 
this method has to be accurate as stopping a system due to a false positive is costly 
and it must be reactive so that the failure is detected before too many damages were 
done. In order to react to the fault, one may either (1) wish to optimise the behaviour 
of a system in the delay before the occurrence of a fault |EMTl'6| . which is particularly 
useful for systems which components are automatically replaced on a regular basis, 
thus hopefully before the occurrence of any fault, or (2) try to detect the fault. As 
one wants to react quickly to the fault, predicting its occurrence before the system 
even enters a faulty behaviour would be very efficient. This view is the one studied in 
prediction problems |GL09j . However, enabling prediction is a very strong requirement 
for a system. Detecting the fault a posteriori is more likely. The study of diagnosis 
raises two important issues: deciding whether the system is diagnosable which is called 
diagnosability and, in the positive case, synthesising a diagnoser possibly satisfying 
additional requirements about memory size, detection delays, etc. In the discrete event 
system context, diagnosis was first defined for finite systems such as partially observable 
Labelled Transition Systems |SSL + 95 then was extended to numerous more complex 
models ( e.g. Petri nets |CGLS12l 1BHSS18] . pushdown systems [MP09 j. etc.) and 
settings (e.g. decentralised [DLTOO], distributed |HC94| 1. Also, several contributions, 
gathered under the generic term of active diagnosis, focus on enforcing the diagnosability 
of a system |SLT981 ITT071 lGT08l IGP09] . 


Useful techniques. By observing the previously mentioned works on diagnosis, it 
appears that some methods and results are recurrent. Let us mention and explain some 
of them here. 

Diagnosability is an hyper property, it cannot be checked by analysing every run of 
the system separately. On the contrary, some runs are faidty (contain the fault) while 
the others are correct and we want to compare the observations triggered by faulty 
runs to the ones produced by correct runs. A key object (e.g. see IJHCKOi] 1YL02] ) 
used to decide diagnosability is the twin-plant: a new model is built by making the 
product of the initial model with itself. A run of the twin-plant consists of a pair 
of runs of the model. As one wants to compare the observations of the runs, the 
product is made so that the two runs that are followed simultaneously have the same 
sequence of observations. This way, one can determine if there exist two runs with 
the same sequence of observations and some appropriate properties only by checking 
a single run of the twin-plant. Another often used construction (see [SSL + 95j i is the 
belief construction. This construction can be seen as an expanded twin-plant or as a 
form of determinisation of the model: instead of following a pair of runs, the belief 
automaton instead follows every possible run. More precisely, we keep every state that 
could be reached with the current sequence of observations, sometimes enhanced with 
some additional information. This construction is more useful than the twin-plant as 
it keeps much more information, however it is of exponential size w.r.t. the size of the 
original model while the twin-plant is only quadratic. The belief automaton by itself 
gives some information, but can also be used to enrich the initial model. For example, 
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assume the belief automaton of a stochastic model was built, one can now build the 
product of this model with its belief automaton |Var99l lBK08| . The result, thanks 
to the determinism of the belief automaton, has the same stochastic behaviour as the 
original model, however configurations now contain two information: (1) the current 
state of the associated run in the initial model and (2) the current belief, i.e. the set of 
states that could be reached with the current sequence of observations. 

Enriching a model this way is very useful to apply model-checking results. Model 
checking consists in, given a model of a system, verifying if it satisfies a property, 
often given by a logical formula. The two most famous logics used are LTL (linear 
temporal logic) |Pnu77| and CTL (computational tree logic) |Eme90| . The first one 
focuses on properties of individual run while the second is mostly interested in branching 
properties. As diagnosis is not a branching property we only discuss LTL here. In LTL, 
one can encode formulae about the future of a run, e.g., a condition will eventually 
be true, a condition will remain true until another one becomes true, etc. The basic 
components of a formula are propositional variables whose truth value depends, in 
our framework, on the current state of the model. The more information is contained 
within a state, the more precise the use of propositional variables can be. How to 
verify that a “simple” model satisfies an LTL formula is known for a long time |Var9(iQ 
Complications occur however when the model is more complex or when the property one 
wants to check requires more expressive power than what LTL can offer. For stochastic 
specifications, LTL was extended to pLTL. The extension allows to quantify the measure 
of the paths satisfying a given LTL formula (the probabilistic operator cannot be nested 
in the formula). Verifying these formulae is more difficult in terms of complexities. It 
has been studied both for finite systems [CY95| or infinite ones |EY12l . One can refer 
to |BK081 Chapter 10] for details about the model checking of probabilistic systems. 
One important point is that the main source of the complexity of the algorithms is 
the size of the formula. For example, in |EY12| . the qualitative model checking of a 
recursive Markov chain (a model of infinite-state stochastic system) is PS PACE in the 
size of the model (and can drop to PTIME under some restrictions) but is EXPTIME in 
the formula. When studying diagnosis, most problems can be expressed with a simple 
and fixed formula, which means that the part of the complexity depending on then size 
of the formula is not our concern. 

Another set of techniques that can be used are the results known for POMDP. 
They have two main interests. First, when studying diagnosis on active systems, some 
problems can be translated into POMDP problems for which there exist efficient algo¬ 
rithms (as done in |BFH + 14| b The second interest of POMDP comes in fact mostly 
from probabilistic automata (PA), which are a subclass of POMDP. Many problems are 
known to be undecidable for PA |Paz711 IGOIO] and due to the simplicity of the PA 
model, these problems are often easier to use to prove undecidability than problems 
for POMDP (such as the policy-existence problem under the infinite-horizon average 
reward criterion |1MHC03] ). 

1 One technique is to obtain a Biichi automaton that is equivalent to the model and another one 
that is equivalent to the negation of the property. The intersection of the two non-deterministic Biichi 
automata is empty if the model satisfies the property. 
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Finally, as there exist many problems using partial observation, the first thing to do 
when studying a new notion is to check if there already exists a similar problem for which 
an analysis was realised. In the positive case, one only has to establish a translation. A 
relevant example of this was mentioned earlier when we stated that |CK14] was used to 
solve a monitoring problem. The authors of |CK14| established a polynomial algorithm 
in order to determine if the distance of the language of two labelled Markov chains 
is equal to 1. In other words, to decide if, almost surely, given an infinite observed 
sequence, one can determine which labelled Markov chain emitted it. The algorithm 
relies on the existence of algorithms to detect when two systems have exactly the same 
language (distance 0) and that if the models are not at distance 1, then a “part” of 
them is at distance 0. While diagnosis focuses mostly on finite runs and this problem 
considers infinite runs, strong links can be identified. 

Challenges and objectives. As our global goal is to perform model-based verifi¬ 
cation, the first question that needs to be tackled in this thesis is the choice of the 
formalism. This formalism has to include probabilities as we want to be able to quan¬ 
tify the specification. But some points are still open: we must determine whether the 
model incorporates non-determinism, represents infinitely many states or expresses ef¬ 
ficiently concurrent behaviours for example. Moreover, we intend to work on partial 
observation problems. This requires the model to select what observation is associated 
with a run. 

Our second issue lies in the choice of the problems to focus on. Many notions of 
diagnosis have been defined over the years with different goals in mind. We have to 
find a set of appropriate qualitative/quantitative diagnosis notions that encompasses 
the important, already known, notions that focus on realistic relevant issues, and that 
is coherent as a whole. Moreover, the formal definitions of the problems we work on 
must be carefully chosen. Indeed, mixing partial observation, probabilities and control 
quickly leads to undecidability results (as is the case for PA, mentioned earlier). A 
slight modification of the definitions may strongly modify the complexity. For example 
we will see a case, where inverting two quantifiers turn a problem of complexity PTIME 
to an undecidable one. 

Once the properties are defined, our goal is to establish precisely the complexities 
of verifying if the model satisfies the chosen specification. We are also interested in 
determining how to modify the system so that it satisfies the properties. This gives two 
main approaches, a passive one associated with observation and an active one associated 
with observation and control. 

Outline. This thesis is organised as follows. 

• In Chapter [2j we introduce notations useful all along the document. While it 
does not contain any result per se, it includes the definitions of the notions of 
diagnosis that we introduced. The choice of appropriate definitions is already a 
contribution. In the second part of this chapter, we present a state of the art on 
the diagnosis problem. 
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• In Chapter [3j we realise a semantical analysis of the problems of diagnosability 
we defined in Chapter [2] More precisely, we first present the notion of diagnoser, 
i.e. the function realising the diagnosis, and prove the equivalence between the 
existence of a diagnoser and the diagnosability of a system. We also establish 
the relations between the various notions of diagnosability and, when possible, 
characterise these notions. The semantical analysis of a problem as done here is 
very important as understanding the problems is the first step to solving them. 
This chapter is based on [ BHL14 ; ! BHL16a1 lBHL16bj . 

• In Chapter [4j we focus on our simplest model, representing finite stochastic sys¬ 
tems. Using the finiteness, we strengthen our characterisations of the diagnosabil¬ 
ity notions and use them in order to establish algorithms to decide the problems 
when possible or to prove undecidability in the opposite case. We also show 
how to build diagnosers using finite memory. This chapter develops contributions 
from |BHL14l [BHLl6a] . 

• In Chapter |5j we turn to systems with infinitely many states. We cannot use the 
characterisations obtained in Chapter[4j but the results of Chapter [3] still hold. We 
study different models and clearly observe the increase in difficulty compared to 
finite-state models. We still manage to obtain decidability results for one model. 
This chapter extends |BHL16b| . 

• In Chapter [6j we consider controllable systems. Using the control, one can ensure 
properties for the system. However, controlling the system with one objective 
in mind can have negative side-effects. For example, ensuring diagnosability can 
increase the likelihood of faults within the system. We are therefore interested in 
combining multiple objectives, one of them being diagnosability. This chapter is 
based on [ BHL17b j. 

• In Chapter [TJ we instead focus on another partial observation problem: opacity, 
which, on many aspects, appears like a dual of diagnosability. We introduce the 
notion and explain the impact opacity has on the choice of the framework: we 
consider here active systems as is done in Chapter [6j however the type of control 
is different as the controller is not interpreted in the same manner. We define 
multiple measures of opacity and explain how to maximise or minimise them 
when possible. The results of this chapter were published in |BHL17a] , 

Other works. In order to limit the number of frameworks and problems to define and 
to give a better coherence to the thesis, some of the works we realised are not detailed 
in this document. We give a short description of these results here. 

The results of |BHL14| serve as a foundation to our analysis of diagnosability. They 
are thus, for the most part, necessary for this thesis and are therefore developed in 
Chapter [3] and Chapter [I] However, this work also contains contributions on prediction 
and prediagnosis. Prediction describes the ability to detect the fault before its occur¬ 
rence. It had been shown to be in NLOGSPACE for logical systems |CL09j . While using 
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probabilities usually increases the difficulty, we gave an NLOGSPACE algorithm for the 
prediction problem in stochastic systems. The authors of JCKl o| present a similar result 
with a notion called “prognosis”. While prediction is limited to the detection of faults 
before their occurrence, diagnosis is itself limited to their detection after the occurrence. 
That is why we also introduced prediagnosis where one is allowed to either predict or 
diagnose the fault. Any predictable or diagnosable system is thus prediagnosable, but 
the converse does not hold. We showed that prediagnosability (z.e. the problem of 
deciding if a system is prediagnosable) is PS PACE-complete. 

For the diagnosability notions studied in this thesis, faults are permanent: once a 
fault is triggered, any following behaviour is faulty. However, one may want to consider 
faults that are only temporary. For example, a model could contain the possibility 
of a reparation. This raises many new problems: one may wish to detect the fault 
before the reparation, to count the number of faults occurring within the system, etc. 
These questions were investigated in |FHLM18| in a non-stochastic setting. While 
diagnosability of permanent faults is known to be in NLOGSPACE for logical systems, 
we showed that with repairable faults, it becomes PS PACE-complete. We also discussed 
multiple methods to count faults and presented among other things an NLOGSPACE 
algorithm to decide if one can count the number of faults while having a delay of at 
most one count of fault. 

One of the non-stochastic, framework where diagnosability was studied is Petri nets 
(PN) [CGLS121 IBHSS18] . For bounded PN, the usual method to solve diagnosability is 
to build the reachability graph of the net, which is an automaton structure representing 
the behaviour of the PN and then to use existing results on this kind of models jJHCKOli 
IYL02] , The issue with this method is the size of the reachability graph. In order 
to face this issue, many works try to abstract the graph in order to reduce its size 
while keeping the relevant information. In a partial observation setting, this led to 
the introduction of the basis reachability graph [CGS09| . In |LGS18j . we showed how 
to extend this abstraction to unbounded PN, calling the new object “Basis coverability 
graph”. We established some results about the properties of the basis coverability graph 
and explained how to use it to solve diagnosability of unbounded PN. 
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Chapter 2 

Preliminaries 


The first step of the theoretical analysis of a problem is the definition of the framework. 
The chosen framework possesses different properties depending on the specificities of 
the system one wishes to study. Some frameworks are thus better adapted to represent 
concurrency, to express infinite-state systems... In the first section of this Chapter, we 
introduce the main definitions that are used throughout the thesis. More precisely, in 
Subsection we recall some definitions and results of descriptive set theory. We then 
define in Subsection 1.2 the main probabilistic model used in this document and define 
a probabilistic measure. In Subsection |1.3| we explain how partial observation can be 
added to this model. These definitions give a framework for various problems linked to 
partial observation. In this section, we also give definitions related to diagnosis, which is 
the main partial observation problem studied during this thesis. Diagnosis corresponds 
to a family of questions, focusing on the identification of a faulty behaviour within the 
system. We explain in Subsection 1.4 how the notion of fault can be formalised in 
our model. Finally, in Subsection 1.5 we discuss different notions of diagnosability. 


These notions are used to capture when and how the faulty (or correct) behaviour of 
the system must be identified, and with which accuracy. Section [2] finally presents a 
state of the art on diagnosis. 


1 Framework 

Let us start with a few general notations. We denote by N the set of natural numbers, 
Q the set of rational numbers and M the real numbers. For a finite alphabet £, we 
denote by £* (resp. the set of finite (resp. infinite) words over £. e represents the 
empty word. 

1.1 Descriptive set theory 

Descriptive set theory |Mos801 Chapter 1] defines and studies classes of “well-behaved” 
sets. These sets having good properties, they have applications in many areas. In this 
thesis, they have two main applications. First, they are used to evaluate the complexity 
of some problems. This is achieved using a hierarchy ranking these sets: the higher a 
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set is in the hierarchy, the more complex it is. Therefore if we can express a property 
with a set, the problem complexity can be related to the set complexity. Secondly, these 
sets form a building block of the formal definition of the stochastic behaviour of our 
model. 

Let us first recall some standard facts about Borel sets. 


Definition 2.1. Given a space X, the set Y is a topology on X if 
• X eY andQeY, 


• Y is stable by union, i.e. given (Oj)j g / a family of elements of Y, Uie/Oj E Y, 

• Y is stable by finite intersection, i.e. given (0,)j g / a finite family of elements of 
Y, n ie iOi E Y, 


We call the sets in Y the open sets. 

Example 2.1. Consider the space of infinite words over two letters {a, b} u . On such a 
space, the usual topology uses the notion of cylinder. Given a finite word w of {a, b}*, the 
cylinder of w is the set of infinite words extending w, Cyl(tu) = w{a, b} u . One obtains 
a topology by choosing the set of cylinders as its basic components and by adding the 
sets created through union and finite intersection, {a, b}^ belongs to this topology as it 
is the cylinder of e, the empty set can be obtained as the intersection of the cylinders of 
‘a’ and of ‘b’. 

Under this topology, the set of words containing at least k ‘a’, for k E N is an open 
set as it is a countable union of cylinders. 


Given a space X and a topology Y on X, we define the Borel hierarchy (represented 
in Figure 2.1) as the three classes of sets £®, II® and A® obtained inductively by 


£® = Y is the set of open sets; 

Vn > 1, a set belongs to II® if its complement belongs to £ 


Vn > 2, O is in £® if there exists a family (Oj)j g / of elements of II®_ 1 such that 

o = u i£iOi] 


Vn>l, A® = n° n s®. 


The sets in II® are called closed, the sets £® and II® are respectively called F a and Gs■ 
A Borel set is a set belonging to some level of the Borel hierarchy. The set of Borel sets 
is denoted 03. 


Example 2.2. Continuing Example \2.1\ the set composed of the single word aP is 
closed. Indeed, its complement is U ng f!jCyl(a n 6) which is an open set. The set of words 
with infinitely many ‘a’ is neither open, nor closed. It however belongs to Gs■ Indeed, 
it is the complement of the set of words ending by b u . As this set of words is an infinite 
union of closed set (each closed set containing a single infinite word), it belongs to F a , 
thus its complement is a Gs set. 



Framework 


37 


A 


o 

1 



A 0 


S 

\ 


n u 


\ 


A 0 

■^a+l 


Figure 2.1: Representation of the Borel hierarchy. 


In our models, we need to measure the probabilities of certain events. This is done 
thanks to Caratheodory’s extension theorem |ADD99| . Before stating this theorem, we 
need to introduce some definitions used in measure theory. 

Definition 2.2. Given a space X, a ring of sets R of X is a subset of the powerset of 
X satisfying: 

• 0 E f?; 

• R is closed under pairwise union, \M, B £ R, AU B £ R; 

• R is closed under relative complements, \/A, B £ R, A \ B £ R. 

Definition 2.3. Given a space X, a cr-algebra S of X is a subset of the powerset of X 
satisfying: 

• (be S; 

• S is closed under countable union, £ S, e S; 

• S is closed under complement, MA e S, S \ A e S. 

By definition, a cr-algebra is a ring of sets. Observe that the set of Borel sets IB is a 
cr-algebra. More precisely, it is the cr-algebra generated by the open sets (i.e. it is the 
smallest cr-algebra containing the open sets). 

Definition 2.4. Given a ring of sets R on a space X, a pre-measure p on R is a 
function p : R i— > [0, +oo] such that: 

• m(0) = °; 

• for all countable family of sets of R pairwise disjoint, (Mj)jgN, we have 

( oo \ oo 

An j = p{A n ). 

n=1 / 72—1 


A pre-measure p is called cr-finite if there exist a countable number of sets A\,Ai--- £ R 
such that X = U/^Li an d for all k £ N, p(A j.) < oo. 

If R is a cr-algebra, then p is called a measure. 
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A measure p is called inner regular if for every set E, we have 
n(E) = sup{p(F) | F C E A F is a closed set}. 


In the current work, we require a measure, yet only a pre-measure can efficiently be 
defined. Fortunately, Caratheodory’s extension theorem allows to bridge the gap. 


Theorem 2.1 ( |ADD99j ). Let R be a ring on a space X, p be a pre-measure on R. 
There exists a measure p' extending p on the a-algebra generated by R. Moreover, if p 
is a-finite, then p' is unique and also a-finite. 


Example 2.3. Let us continue Example 2.1 and consider the ring of sets R generated 
by the topology based on the cylinders. We define the pre-measure p on this ring as the 
only pre-measure satisfying Vn £ N, w £ {a,b} n , p(Cy\(w)) = 

One could interpret the represented system as an infinite number of coin flips, each 
result (‘a’ or ‘b’) having | probability, p gives the probability of cylinders (a given finite 
number of flips) and finite unions of them. 

The pre-measure p is a-finite as p({a, b}^) = 1. According to Caratheodory’s exten¬ 
sion theorem, there is thus an imiqiLe measure p! extending p on the a-algebra generated 
by R: the Borel sets. Observe that p' is inner regular. 


1.2 Probabilistic Labelled Transition Systems 

We now define the model that represent the system. The choice of the model depends on 
the properties one wishes to have. Petri nets [Dia09| for example, efficiently represent 
concurrent systems. Another possibility is to use automata structures as in the seminal 
work of |SSL + 95| formally defined by: 

Definition 2.5. A labelled transition system (LTS) is a tuple A = {Q, qo,T,,T) where: 

• Q is a coimtable set of states with qo £ Q the initial state; 

• E is a finite set of events; 

• TCQxFxQ is a set of transitions; 

Informally, the states of Q represent the different configurations the system can be 
in, an event of E is an action that can be taken by the system (sending a request, 
activating a component...) and T describes how this action affects the system. 

Formally, we write q q' when there exists a transition ( q , a, q') £ T; this transition 
is then said to be enabled in state q. We assume all LTS we consider are live, i.e. in 
every state of the LTS at least one transition is enabled. This ensures the system will 
not reach a deadlock position and stop activating events. A run p of an LTS A is a 
(finite or infinite) sequence p = q^a^qi ... such that for all i > 0, qi £ Q, ai £ E and 
when qi + \ is defined, qi —b qi+\. A run thus represents the evolution of a system over 
time. The notion of run can be generalised, starting from an arbitrary state q. Given 
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an LTS A, we write fl A for the set of all infinite runs starting from q q. We only write Q 
when the LTS A is clear from context. When it is finite, p ends in a state that we denote 
last(/o) and its length , denoted by \p\, is the number of events occurring in it. Given a 
finite run p = qoaoqi... q n and a (finite or infinite) run p' = q n a n q n + 1 ... starting in 
last(p), we call concatenation of p and p' the run pp' = qoaoqi... q n a n q n+ \ .... The 
run p is then a prefix of pp', which we denote by p A pp'. The cylinder generated by a 
finite run p consists of all the infinite runs that extend p\ Cyl {p) = { p’ E Fl \ p A p 1 }. 
The sequence associated with p = qaoqi ... is the word a p = aoOi..and we write 
indifferently q =>■ or q =A (resp. q => q' or q =!■ q') for an infinite (resp. finite) run 
p starting in q (resp. and ending in q'). A state q is reachable (from the initial state 
go) if there exists a run p such that qo q, which we alternatively write qo => q. The 
language of an LTS A consists of all infinite words that label runs of A and is formally 
defined as LA (A) = {a E TF \ 3 qo =§>}. A bottom strongly connected component 
(BSCC) of an LTS is a strongly connected component from which no state outside of 
the BSCC are reachable. 

Example 2.4. Consider the LTS represented in Figure \2J^ It contains three states, qo 
is the initial state, representing the machine waiting to receive an order. When such an 
order occims the run takes the transition labelled by the ‘coin’ event and enters the second 
state. In this second state, the machine is preparing the coffee, it has the possibility to 
add sugar, and after a certain amount has been added, it returns to its initial state by 
giving a coffee. In this operating state, the machine can also commit an error, event 
{’, leading to a faidty state f±. In f\, the machine cannot serve coffee any more and 
sends an ‘out of order’ signal. A normal use of this system by a consumer is given for 
example by the run p = qo coin q± sugar q\ coffee qo- 

sugar out of order 

com 



Figure 2.2: An LTS representing a coffee machine, qo is the initial state, which is 
represented by the incoming arrow. Transitions between two states are labelled by the 
event associated with the transition. 


In order to represent the unpredictability of the environment and to neglect events 
that have a null probability of occurring, we want to represent stochastic behaviours. To 
do so, the model must be enriched using probabilities. More precisely, this is achieved 
by adding a probability matrix indicating how the transitions, labelled by events, are 
randomly chosen. 
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Definition 2.6. A probabilistic labelled transition system (pLTS) is a tuple A = 
(Q,qo,T,,T, P) where: 

• A = ( Q , qo , E, T) is an LTS; 

• P is the transition matrix from T to Q>o fulfilling for all q G Q: 

^2 P [q,a,d] = 1 • 

{q,a,q')eT 


The LTS A is called the underlying labelled transition system of A. 


Note that since we assume the state space to be at most countable, a pLTS is 
by definition at most countably branching: in every state q, only countably many 
transitions are enabled, so that the summation X](q a q')eT P[<Z> a i l/] well-defined. 
The dehnitions introduced for LTS are naturally lifted to pLTS. Given a countable set 
Z. a distribution on Z is a mapping p : Z —> [0,1] such that YIzgZ L( z ) = 1- The 
support of p is Supp(/u) = {z € Z \ fj,(z) > 0}. If Supp(/i) = { z } is a single element, fj, 
is a Dirac distribution on z written l z . We denote by Dist(Z) the set of distributions 
on Z. The transition matrix defines in every state q a distribution on the transitions 
whose support are exactly the transitions enabled in q. 


Example 2.5. Consider the pLTS represented in Figure pO[ Its underlying LTS is 
represented in Figure 2.2. The difference is thus that probabilities were added on the 
transitions so that the sum of the probabilities exiting any state is equal to one. The 
run p that was described as being a normal use of the system in Example \2.I\ can now 
be associated with a probability, which is the product of the probability of the events it 
triggered. Here, 1 x 0.29 x 0.7 = 0.203. This residt, being low, seems to point out that, 
in this representation, most consumers do not take exactly one unit of sugar in their 
coffee or do not even get their coffee. 


sugar, 0.29 out of order, 1 



Figure 2.3: A pLTS representing a coffee machine. The probability of a transition is 
given next to the event labelling it. 


We now use the probabilities within the system to formally define the probability 
measure that are used on runs. This is done using the descriptive set theory presented 
in Subsection 0 The construction of the measure uses the Caratheodory’s extension 
theorem, similarly to what is done in Example |2.3[ 
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Here, the set space X we are interested in is the set of all infinite runs f2. The open 
set Xq is built from the cylinders: Zq is the smallest set containing 0, Q, such that for all 
finite run p, Cyl (p) E Sj, and such that is stable by union and finite intersection. The 
complement of a cylinder is a finite union of cylinders, the complement of an (potentially 
infinite) union is an (potentially infinite) intersection and the complement of a finite 
intersection is a finite union. A set F is thus closed if and only if F = fj ngN O n where O n 
is a union of cylinders. Therefore an F a set F can be written as F = UmeN flneN O m ,n 
where O m)n is a union of cylinders whose associated paths have length n. Without loss 
of generality, the sequence of closed sets may be chosen as a non-decreasing sequence. 
The cylinders are thus used as the basis to build a Borel hierarchy on Q. 

Given a pLTS A, in order to have a probabilistic measure spanning all the sets of 
infinite runs of the Borel hierarchy generated by the cylinders, we define a pre-measure 
P _4 on the ring of sets generated by the open sets by: for all finite run qoaoqi ■ ■ ■ q n , 

IP^(Cyl(goao9i • • • Qn)) = Pfeo, a 0 , qi\ ■ ■ ■ P[q n -i,a n -i,q n ] 
and for all finite sequence of pairwise disjoint cylinders A \,..., A n . 

IPa(Ui=1,. = "y ' ^A^i) ■ 

According to Caratheodory’s extension theorem, this pre-measure can be extended 
uniquely on the whole cr-algebra generated by this ring. We still write P _4 for the 
obtained measure. Moreover, when A is fixed, we may omit the subscript. To simplify, 
for p a finite run, we sometimes abuse notation and write P(p) for P(Cyl(p)). If R is 
a (countable) set of finite runs such that no run is a prefix of another one, we write 
P(i?) for w hi c h is consistent since all intersections of associated cylinders 

are empty. 


1.3 Partial observation 


In partial observation problems, one considers an observer of the system (a user, an 
attacker...) who perfectly knows the model, yet only partially observes its behaviours. 
In this case, one needs to formalise which information can be observed. This allows to 
associate with every run of the system, an observation or sequence of observations. 

The first question is to determine what part of a run gives an information: events, 
states or both. In fact, these three options are equivalent in terms of expressibility. 
Thus, for every problem, one selects the option that is the more efficient at representing 
the specificities of the problem. For example, when studying diagnosis, we try to detect 
a faulty action of the system. As this faulty action is an event, observations are put 
on events. This is the option we follow in most chapters of this thesis, see Chapter [7J 
for a model with observations on states. 
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Events represent actions taken by the system. A formalisation of the observation 
could be made by distinguishing internal and external actions. This would mean that 
some actions occur within the system and are thus unobservable while others are done 
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publicly and are thus observable. Formally, one partitions the set of events S into two 
disjoint sets E 0 and E u , the sets of observable and unobservable events, respectively. 

Example 2.6. Consider the pLTS of Figure \2.3\ Clearly, introducing a coin, adding 
sugar to the cup and receiving the coffee are observable actions. The other two are more 
subject to discussion. Depending on the fault and the sensors within the system, f could 
be considered observable or unobservable. Indeed, if f stands for the explosion of the 
machine, it would clearly be observable, however if f is an internal error, detecting it 
requires the existence of an appropriate sensor within the system. The status of the 
last event also depends on which fault occurred. For our example, let us assume f is 
unobservable, but the failure of the system is detected, allowing the machine to publicly 
send an ‘out of order’ message. In this case, the rim “qo coin q\ sugar q\ f f\ out of 
order” produces the sequence of observations “coin sugar out of order”. 

A more sophisticated method than this partition would be to equip the pLTS with 
a mask function. This mask function associates every event with an observation, taken 
from an observation alphabet. This function can map an event to e meaning that the 
event is unobservable or project multiple events onto the same observation, making 
them indistinguishable. When using a mask function, E 0 is the observation alphabet 
and T, u is the set of unobservable events (he. events which observation is e). 

Example 2.7. Consider the pLTS of Figure \2. 3| again. We use the observation alphabet 
E 0 = {coin, coffee, beep} and the mask function V such thatV(coin) = coin, V {coffee) = 
coffee, V(f) = e and V(sugar) = V(out of order) = beep. Here, we see that two of the 
events are indistinguishable as they share the same observation beep. When a beep is 
produced, a user does not know whether the machine is out of service or if it is adding 
more sugar. In other words, the infinite runs “qo coin q\If\(out of order fi) u ” and “qo 
coin qi(sugar q\)“ J ” share the same observation sequence “coin sugar beep‘d 

Observe that the mask function setting generalises the partition discussed above. In¬ 
deed, the partition is mimicked by the mask function which projects every unobservable 
event to e and every observable event onto itself. We now define multiple notations using 
the mask function formalism. Due to the previous remark, these definitions can easily 
be applied to the partition setting. In the future chapters, we mostly use partitions for 
simplicity. We state explicitly when we use mask functions. 

Given an observation alphabet, a mask function is a mapping V : E —> E 0 . It is 
extended to words from E* inductively by: V(e) = e and V(aa) = V(a)V(a). We 
write |oj 0 for the observable length of a, that is \V(a)\. The observable length of a 
run p, denoted \p\o, is the observable length of its associated sequence. Given a run 
p and its sequence a p we sometimes use V(p) for V(a p ). When a is an infinite word 
over E, its projection (resp. observable length) is the limit of the projections (resp. 
observable length) of its finite prefixes. Given a £ E 0 , \cr\ a is the number of occurrences 
of a in a. As usual the mask function V is extended to languages: for L C E* U E w , 
V(L) = {V(a) \ a L}. 

With respect to the mask function V, a pLTS A is said convergent if there is no 
infinite sequence of unobservable events from any reachable state: ^(A) D E*E^ = 0. 
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When A is convergent, for every a £ C UJ (A ), 'P(cr) £ In the rest of the thesis we 
assume that pLTS are convergent. We refer to a sequence for a finite or infinite word 
over £, and to an observed sequence for a finite or infinite word over £ 0 . The projection 
of a sequence onto £ 0 is thus an observed sequence. The prefix of length n £ N of an 
observed sequence w is denoted w< n . 

We now define the notion of signalling runs. They correspond to the finite runs 
which last event was observable (i.e. finite runs qoaoQi ■ ■ ■ a n _i q n such that V(a n - 1 ) A 
e). Signalling runs are precisely the relevant runs w.r.t. partial observation issues 
since each observable event provides additional information about the execution to an 
external observer. In the sequel, SR(„4) denotes the set of signalling runs of the pLTS 
A, and SR n (.A) the set of signalling runs of observable length n. The pLTS is dropped 
from the notation when it is clear from context. Since we assume that the pLTS are 
convergent, for every n > 0, SR n is equipped with a probability distribution defined 
by assigning measure P(p) to each p £ SR n . Given p a finite or infinite run, and 
n < \p\ 0 , Pin denotes the unique prefix of p that belongs to SR n . For convenience, the 
empty run go is defined as the single signalling run of null length. For an observed 
sequence w £ £*, we define its cylinder Cyl(ic) = wYff and the associated probability 
P(Cyl(u>)) = P({p £ 17 | V{pi\ w \) = tc}) = P({p £ SR^ | V(p) = tc}), often shortened 
as P(iy). 

1.4 Fault and ambiguity 

We now give definitions and notations for a partial observation problem, which is par¬ 
ticularly of interest to us, diagnosis. Diagnosis focuses on the detection of a special 
unobservable event called the fault f £ £ u thanks to the observations received from 
the system. Let us now classify runs depending on whether they contain a fault or 
not. A run p is faulty if its associated sequence a p contains f, otherwise it is correct. 
For n £ N, we write F n (resp. C n ) for the set of infinite runs whose signalling prefix 
of observable length n is faulty (resp. correct). We further define the sets of all finite 
faulty and correct signalling runs F and C and the sets of infinite faulty and correct runs 
Foo = UneN Fn and Cqo = UneN Cn- A run /0 is a minimal faulty run if it is a faulty run 
and there does not exist a prefix p' of p that is a faulty run. We write, for all n £ N, 
minFn for the set of minimal faulty runs of length n and minF = UneN m inF n for the 
set of all minimal faulty runs. 

Given two states q and q' and an observation a £ £„, we write q => a q' if there exists 
a run p = q^a^qi... q n with go = Q, qn = q ', p €. SRi and V(p) = a. We also write 
q q' (resp. g =>“ g') if there exists a faulty (resp. correct) run p = q^a^qi ... q n with 
go = g, q n = q ', p € SRi n F (resp. p £ SRi n C) and V(p) = a. 

Except explicit mention of the opposite, we assume that the state space Q of A 
is partitioned into correct states and faulty states: Q = Qf l±l Q c such that faulty 
(resp. correct) states, i.e. states in Qf (resp. Q c ), are only reachable by faulty (resp. 
correct) runs. This can be done without loss of generality. Indeed, considering a pLTS 
A = (Q , go, £, T, P), we can build the pLTS A' = ( Q (go, _L), £, T', P') where: 

. Q' = Q x {1, T}, 
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• ((q,i),a, (q', j)) G T' if ( q,a,q’) G T, and either a / f and i = j or a = f and 

j = T, 

• for ((<?,*), a, (q',j)) G T', P'((g,f),a, (g',j)) = P(g,a,g'). 

Denoting Qf = Q X {T} and Q c = Q x {_L} , the pLTS „4/ has the same behaviour as 
A and verifies the partition mentioned above as a run enters Qf if and only if its last 
transition was a fault and can never go back to Q c . 

While the correct or faulty status of the current run may not be known to the 
observer, the observed sequences carry some information about them. An infinite (resp. 
finite) observed sequence w G Eff (resp. £*) is called ambiguous if there exists a correct 
infinite (resp. signalling) run p and a faulty infinite (resp. signalling) run p> such that 
V(p) = V(p') = w. Otherwise, it is either surely faulty, or surely correct depending on 
whether V^ifw) 0 SR C F or V~ 1 (w) 0 SR C C. A run is ambiguous, surely correct 
or surely faulty if its observed sequence is ambiguous, surely correct or surely faulty 
respectively. We write Sfoo (resp. Scoo) for the set of infinite surely faulty (resp. 
correct) runs. In addition Sf n (resp. Sc n ) is the set of infinite runs whose signalling 
prefix of observable length n is surely faulty (resp. correct). 

Example 2.8. Consider the pLTS of Figure associated with the mask function V 
such thatV( f) = £, V(out of order ) = V(sugar) = beep and every other event is projected 
on itself. 

First observe that this pLTS satisfies the partition between faulty and correct states. 
Indeed, a run ends in f± iff it is faidty. 

The observed sequence “coin beep” is ambiguous as it can be generated by the correct 
run “go coin q\ sugar q\ ” and the faulty signalling run “go coin q\ f f\ out of order 
fi”. Extending this observed sequence with other observations of beep maintains the 
ambiguity. Extending it with ‘coffee’ however makes it surely correct. There does not 
exist any surely faulty observed sequence in this pLTS. 

1.5 Which diagnosis for pLTS? 

The goal of diagnosis is the automatic detection of the fault event. This detection is 
performed by a diagnoser, a function observing the system and giving its verdict. For¬ 
mally, a diagnoser is a function D : E* —>• {?,T,_L} assigning to every finite observed 
sequence a verdict. Informally, when a diagnoser outputs ? it does not provide any 
information, while T means that the diagnoser announces a fault and _L that the di¬ 
agnoser provides some information about the correctness of the current run. Multiple 
notions of diagnoser, and thus of diagnosis, can be defined depending on the properties 
that we require. In logical systems, three main features of the diagnoser are considered: 
verdict, correctness and reactivity. Verdict specifies the nature of the information the 
diagnoser provides along the run: it may only be related to detection of faults or may 
also assert that (some prefix of) the run does not include a fault. Correctness specifies 
that when the diagnoser outputs a verdict, this verdict holds. Reactivity expresses the 
regularity at which the diagnoser must provide information about the status of the run. 
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The aim of this section is to define appropriate verdict, correctness and reactivity 
requirements for probabilistic systems. We start with informal explanations that also 
motivate the need for considering different variants of diagnosis. We present these 
variants as decision problems which are intuitively easier to understand and simpler to 
use. We make the link with diagnosers in Chapter [3] 


In seminal works about probabilistic systems, the verdict is limited to fault detec¬ 
tions and the reactivity is usually relaxed by requiring that when a fault occurs, a 
diagnoser almost surely detects it after a finite delay |TT05j . Let us look at the pLTS 
of Figure 2.4 One cannot detect that the run qoi(fia) u is faulty due to the correct run 
qoulqia)^ with same observed sequence a u . However with probability 1, a faulty run 
will produce a ‘6’ and thus almost all faulty runs are unambiguous, so that faults are 
almost surely detected. On the other hand, one cannot provide any information about 
the single correct run qou(q\ a) w since its observed sequence is ambiguous as well as any 
of its prefix. Observe that the notion of ambiguity described here is qualitative: the 
observation of the correct run is considered ambiguous even though the probability to 
be faulty, conditioned on the observation, converges to 0. 



Figure 2.4: Detecting faults but not correct runs. When probabilities are not specified, 
we assume uniform distributions. Dashed edges are used for unobservable transitions. 
For observable transitions, the observation given by the mask function labels the edge. 


In order to examine which verdict could be provided about correct runs, let us look 

The sequence a n is ambiguous. However up to the n — 1 th 


2.5 


at the pLTS of Figure 
observation, all the runs that correspond to this observed sequence were correct which is 
a useful information for instance to restart later the system from a correct state. Along 
the (surely correct) observed sequence a u , the observer can always deduce that longer 
and longer prefixes of the run were correct while never being able to assert that the 
current run is correct. 



Figure 2.5: Detecting correctness for longer and longer prefixes of correct runs. 
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The correctness requirement may be specified in different ways. For an exact di¬ 
agnosis, we ask that a fault can be claimed only when a fault surely happened (as it 
is the case in non-probabilistic systems). However it may be necessary to weaken the 
correctness requirement as illustrated by the pLTS of Figure |2.6[ Since all observed 
sequences are ambiguous no exact diagnosis can be provided. However it is clear that 
when in a long enough observed sequence the ratio between occurrences of ‘6’ and ‘a’ is 
close to 3, the probability that the corresponding run is faulty is close to 1. Let us fix 
any e > 0 and only require that the probability for the verdict to be erroneous should be 
less than e. Then using the strong law of large numbers, (approximate) fault detection 
is possible in this pLTS. 
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Figure 2.6: When approximate diagnosis is necessary. 


To formalise later the correctness of an approximate diagnoser, with every observed 
sequence w S S* we associate a correctness proportion 

r d i \ _ CnSR M | V(p) = w}) 

0,> wt - P({? e SR m | V(p) = u>» ’ 

which is the conditional probability that a signalling run is correct given that its ob¬ 
served sequence is w. 

The standard way to specify reactivity in probabilistic systems for fault detection 
is to require that whatever the minimal faulty run, almost surely the diagnoser will 
output its (faulty) verdict. We may also consider uniform reactivity which strengthens 
reactivity by requiring that the (random) delay is independent of the minimal faulty run. 
More formally, uniform reactivity ensures that given any positive probability threshold 
a > 0 there exists a delay n a independent of the considered minimal faulty run such 
that the probability to exceed this detection delay is bounded by a. 



Figure 2.7: When reactivity cannot be uniform. 
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Let us illustrate these reactivity features with the pLTS of Figure 2.7 for which only 
approximate diagnosis is possible. Fix some e > 0 and consider the minimal faulty run 
qouqi(aqi) m iqf. After a certain number of occurrences of ‘6’ (say n), the correctness 
proportion of the observed sequence a m b n will be less than z and thus the diagnoser can 
output its verdict. However due to the probabilities of an occurrence of ’a’ in correct and 
faulty runs respectively equal to | and . n must depend on m and so this reactivity 
cannot be uniform. This can be mathematically seen through the definition of CorP: 
for n > 1, 

3 m 1 

CorP (aT'"' 


l b n ) = 


4 m-\-n 


^.m+n 


X 16 


1 + if X 4 n ~ 


In order to have CorP(a m 6 n ) < 1/2, one needs n > rn. Therefore uniform diagnosis is 
not possible. 

In order to formalise the different requirements discussed above, we first define 
several sets of runs related to ambiguity. 


Definition 2.7 (Ambiguous runs). Let A be a pLTS, e > 0 and n E N>o- Then: 


• FAmboo is the set of infinite faulty ambiguous runs of A; 

• CAmboo is the set of infinite correct ambiguous runs of A; 

• FAmb n is the set of infinite runs of A whose signalling prefix of observable length 
n is faulty and ambiguous; 

• CAmb rt is the set of infinite runs of A whose signalling prefix of observable length 
n is correct and ambiguous. 


• FAmb^ is the set of infinite faulty ambiguous runs of A whose observed sequence 
of length n, w fulfils: CorP(u;) > e. 

By definition, for all n E N, FAmb(( = FAmb„. Observe that, for all n E N, and 
z > 0, CAmb n , FAmb„ and FAmb^ are open sets, thus measurable. However, CAmboo 
and FAmboo ar e n °t Borel sets in the general case ( e.g. see Chapter [3j Section [3]). 

We propose five specifications of exact diagnosability for probabilistic systems based 
on three discriminating criteria: whether the unambiguity requirement holds for faulty 
runs only or for all runs, whether ambiguity is defined at the level of infinite runs or 
for longer and longer finite signalling prefixes, and whether the delay before detection 


of minimal faulty runs is uniform. These notions are summarised in Figure 2.8 except 
for uniformity postponed to next figure. 


Definition 2.8 (Exact diagnosability). Let A be a pLTS. 

• A is IF-diagnosable z/P(FAmboo) — 0. 

• A is lA-diagnosable i/P(FAmboo l±J CAmboo) = 0. 
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Reactivity 


Finite prefixes 


FF-diagnosis FA-diagnosis 


Infinite runs 


IF- diagnosis I A- diagnosis 


-1-1-► Verdict 

Faulty runs All runs 

Figure 2.8: Summarising the variants of exact diagnosis. 


• A is FF-diagnosable */limsup n ^. 0O P(FAmb n ) = 0. 

• A is FA-diagnosable */limsup ri ^. 0O P(FAmb n l±J CAmb„) = 0. 

• A is uniformly FF-diagnosable if for all a > 0 there exists n a £ N such that for 
all n > n a and all minimal faulty run p £ minF 

P({/»' e FAmb„ + | p | o | p A p'}) < a ■ P(p) . 


Uniform and/or approximate diagnoses are defined for FF-diagnosis as summarised 
We chose FF-diagnosis as it corresponds to the classical notion of diag- 


in Figure 2.9 


nosis. Moreover there is no clear intuition on what would be the meaning of uniformity 
and approximation for the other variants. eFF -diagnosability allows the diagnoser to 
claim a fault when the correctness proportion does not exceed e, and accurate approx¬ 
imate diagnosability denoted by AFF-diagnosability corresponds to eFF-diagnosability 
for arbitrary e > 0. 

Definition 2.9 (Approximate diagnosability). Let A be a pLTS, and e > 0. 

• A is eFF-diagnosable if for every minimal faulty run p £ minF and all a > 0 there 
exists n Pt a such that for all n > n p>a : 


P(Cyl (p) n FAmb^ +Wo ) < a • P(p). 


• A is uniformly eFF-diagnosable if for all a > 0 there exists n a such that for all 
minimal faulty run p £ minF and all n > n a : 

P(Cyl (p) n FAmbf l+|p|o ) < a • P(p). 





State of the art on diagnosis 


49 


Reactivity 


Uniform 
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uniform 

AFF-diagnosis 


uniform 

FF-diagnosis 


Simple 


£ FF-diagnosis 


AFF-diagnosis 


FF-diagnosis 


Correctness 

-<■-*- 4 -►- 

e approximate Accurate approximate Exact 


Figure 2.9: Summarising the approximate variants of FF-diagnosis. 


• A is (resp. uniformly) AFF-diagnosable if it is (resp. uniformly) £FF-diagnosable 
for all e > 0. 


When studying diagnosis, we are interested in the following problems. First, for 
every notion of diagnosability, we want to determine if it is possible to automatically 
decide whether a given system is diagnosable. Moreover, in the positive case, we want 
to establish the exact complexity of the problem. Then, given a diagnosable system, we 
want to build a diagnoser satisfying the corresponding verdict, correctness and reactivity 
features. When possible, we represent the diagnoser using a finite-state automaton to 
express that only a finite memory is necessary. The problems can vary depending on 
the framework. For example, when the system is controllable, we want to decide if one 
can control the system in a way ensuring diagnosability. 


2 State of the art on diagnosis 


Diagnosis of finite LTS. For LTS, diagnosability requires that the occurrence of 
unobservable faults can be deduced after a finite delay from the sequence of observable 
events occurring before and after the fault [SSL + 95] , Using the definitions we introduced 
in this chapter, an LTS is diagnosable iff UneN Dm>rt FAmb m = 0. Diagnosability of 
finite LTS was shown to be decidable in NLOGSPACE [JHCKOlt 1YL02] . The algorithm 


relies on the twin-plant described page 29 An LTS is not diagnosable iff there exists a 
reachable cycle in the twin-plant in which each state is a pair composed of a faulty and a 
correct state. Detecting such a cycle can be done in non-deterministic logarithmic space 
in the twin-plant, which is quadratic in the size of the original LTS, hence the result. 
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Surprisingly, while deciding diagnosability is easy, the construction of the diagnoser can 
require exponential time. This construction is based on the belief construction quickly 


presented page 29 


Diagnosis of infinite LTS. An LTS with infinitely many states must be represented 
by a higher level model in order to be studied. This can be done using Petri nets (PN) for 
example. The semantics of a PN is called the reachability graph and can be represented 
by an LTS. This LTS is finite iff the PN is bounded. Cabasino et al. studied diagnosis 
of both bounded and unbounded PN |CGLS12j . In order to solve diagnosability in 
the unbounded case they first build the verifier net, which is a construction similar 
to the twin-plant, then construct the coverability graph (a finite abstraction of the 
reachability graph) of the verifier net and finally analyse the cycles of the coverability 
graph. This final analysis requires some additional properties of the cycles compared to 
the analysis in the twin-plant for finite LTS. This algorithm however has a complexity 
that depends on the size of the coverability graph, which may be Ackermanian in the 
size of the description of the PN. An algorithm with better complexity was developed 
in |BHSS18] , It still uses the verifier net, but transforms the diagnosability problem into 
an LTL formula and uses model-checking results to obtain an EXPSPACE upper bound. 
Interestingly, this paper also studies opacity, confirming the intuition that opacity and 
diagnosis are two close problems. See |Basl4j for a presentation of the usual techniques 
used for fault diagnosis in PN. LTS with infinitely many states can also be represented 
by pushdown systems. Morvan and Pinchinat showed that diagnosability in the general 
case is undecidable |MP09) . However, for a large subclass called visibly pushdown 
automata, diagnosability can be decided in PTIME. This is done by building once again 
a form of twin-plant, this time making the product of the visibly pushdown automata. 
Thanks to this product, they can define an appropriate Biichi condition on the twin- 
plant so that diagnosability can be deduced from the emptiness of the obtained Biichi 
pushdown automaton (checking the emptiness can be done in PTIME). The restriction 
to visibly pushdown automata, which we discuss in Chapter [5j is required in order to 
build the twin-plant. 


Diagnosis of stochastic systems. When diagnosability was adapted to stochastic 
systems by Thorsley and Teneketzis in |ITT05j . two notions of diagnosability were ini¬ 
tially defined: A-diagnosability and AA-diagnosability. In finite pLTS, A-diagnosability 
corresponds to our uniform FF-diagnosability and AA-diagnosability corresponds to our 
uniform A FF-diagnosability. For A-diagnosability, they gave a necessary and sufficient 
condition based on the belief construction: they first build a diagnoser similarly to 
what was done for logical systems, then test for the recurrence of ambiguous states 
of the belief {i.e. states that can contain both correct and faulty states). The com¬ 
plexity of the algorithm checking this characterisation is not mentioned however. For 
AA-diagnosability, they only give a sufficient condition, leaving the general problem 
open. The questions left opened by |TT05j were tackled by Chen and Kumar who 
gave algorithms with PTIME complexities to answer both diagnosability decision prob¬ 
lems |CK13| . Their algorithm for AA-diagnosability is particularly interesting as they 
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translate the problem into a question of language equivalence (in terms of probability 
of words) for the original pLTS in a specific initial distribution. Unfortunately, these 
two algorithms are erroneous (see Chapter |d] for details on these two problems). 

When a pLTS is not diagnosable, one interesting question that can be raised is 
how far from diagnosability it is. If there is only a very little chance that the fault 
is not detected, the system may be “diagnosable enough”. This direction was studied 
in (ND08| where Nouioua and Dague consider an exact notion of diagnosability and 
wish to measure the probability of the faulty ambiguous runs. To realise this measure, 
they make the product of the pLTS with its belief construction (thus a method that 
we already presented page 29). Then they measure the asymptotic probability to be in 
each state of this product pLTS. As, thanks to the belief, states contain the relevant 
information to determine if they were reached by a faulty ambiguous run, they can 
determine the probability to be faulty and ambiguous at the limit. This approach is 
continued in |BFG17| where Bazille et al. introduce a notion of /c-diagnosability degree, 
which is defined as the probability to detect a fault at most k steps after it occurs, 
conditioned to the occurrence of a fault. They measure this degree by (1) building 
the product of the pLTS with the belief construction and (2) using polynomial time 
algorithms that compute the sum of the probabilities of the runs that reach a target 
state set (in this case, the states which belief component show they were reached by 
faulty ambiguous runs). Using the computation of £:-diagnosability degrees for different 
values of k, they also investigate the average speed of detection of a fault. 


Active diagnosis. One can enrich an LTS by allowing a form of control. This is done 
by introducing non-determinism, that is resolved at every step by a controller. One 
possibility of control is done through restriction of the enabled events: at every step, 
the controller selects a set of observable events X* C X 0 and the next transition taken 
by the LTS is labelled either by an unobservable event or by an observable one which 
observation belongs to X*. Some observable events can also be considered uncontrol¬ 
lable and are enabled no matter the choice of the controller. In this framework, the 
diagnosability of a system depends on the choice of the controller. Finding a controller 
such that the controlled system is diagnosable is the goal of active diagnosis. This 
problem was introduced in |SLT98j . Sampath, Lafortune and Teneketzis then solve 
the question by building the most permissive controller through a complicated iterative 
procedure which complexity is not given (and seems to be doubly exponential). Later, 
a planning-based approach via a twin-plant construction was proposed in (CP09| . The 
exact complexities were finally established in [HHMS131 IHHMS17] . where the active 
diagnosis problem was shown to be EXPTIME-complete and finite-memory controllers 
(most permissive and optimal in memory size) are given. This is done by translating 
the active diagnosis problem into a Biichi game (using a variant of the product with the 
belief) and then solving the Biichi game, which gives an optimal strategy which can be 
translated back into a controller. This analysis was extended to controllable stochastic 
systems [BFHM.4], Here, instead of translating the problem to a Biichi game, Bertrand 
et al. use partially observable Markov decision processes and show that the problem of 
stochastic active diagnosis is also EXPTIME-complete. They also study a safe notion of 
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stochastic active diagnosis where the controller is required to keep a positive probability 
of infinite correct runs. This second problem is then showed to be undecidable in the 
general case and EXPTIME-complete when limited to finite-memory strategies. 

The control on the system is not necessarily related to the events, it can also be 
applied on the observations. The observations of a system are given by sensors. In 
order to detect an event, one needs to have a sensor at the appropriate position and for 
it to be switched on. In |CT08| and |TT07| the authors investigate in slightly different 
frameworks, how to limit the number of sensors needed and how to build a controller 
which chooses at every step which sensor is switched on or off. The main differences 
between the two papers are twofold: (1) |TT07] considers both logical and stochastic 
systems while |CT08| only focuses on logical systems and (2) (CT08| establishes that 
a most permissive finite-state controller can be computed in doubly exponential time, 
using a game-theoretic approach while |TT07| does not give the exact complexity of 
their algorithm. 

See fZT7T5] for a survey on diagnosis mainly describing results for logical systems, 
but discussing also timed, stochastic and active systems. 
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Chapter 3 

Semantical analysis of 
diagnosability 


As explained in Chapter [2j diagnosers observe the system to determine if its behaviour 
is correct or not. They are formalised as functions giving a verdict to each sequence of 
observation produced by a run of the system. There exist many different ways to define 
a diagnoser, depending on the properties a system designer may want. We identified as 
the most important features of a diagnoser its verdict, correctness and reactivity. 

• The verdict determines what information is given by the diagnoser. Consequently, 
changing the verdict literally means modifying the purpose of the diagnoser. For 
example, when testing a car, if a component malfunctions, the company needs 
to detect it. The diagnoser thus only needs to detect the faulty behaviour of the 
system. However, the verdict of a diagnoser could also require to detect the correct 
behaviour of a system. For example, when following online a critical system like 
a power plant, the technicians needs to know that the system is correct. If the 
system can be faulty, they may need to shut it down in order not to take any risk, 
which can have an important cost. 

• The correctness determines if the diagnoser is allowed to make an erroneous claim 
and, if so, how accurate must its claim be. It is of course better to have a diagnoser 
that does not make any error, but this restriction is not always realistic. Consider 
a program simulating dice throws. If the program outputs 4 a high number of 
times in a row, this may be a correct behaviour as such a throw has a positive 
probability. However, the longer this streak of 4 continues, the more likely it is 
caused by a malfunction of the program. 

• The reactivity determines how quickly and how often the diagnoser must output a 
verdict. One possibility would be to require that if a fault occurs, after a bounded 
delay this fault will be detected. This is the reactivity that is often required in non- 
probabilistic systems represented by an automaton |SSL + 95] , This requirement 
is too strong however for probabilistic systems. Indeed, in probabilistic systems, 
one can have for example an event which can occur at every step after a fault 
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with some probability, say 1/2, and which allows the detection of the fault. In 
expectation, the fault is thus detected after two steps, but no bound can be given 
on the maximum delay. This is a situation occurring in every system where, after 
a fault, the system can, with some probability, continue to act normally. When 
studying probabilistic systems, the reactivity requirement must thus be adapted. 
This can be done, for example, by requiring that with probability 1 the fault will 
be detected. This version of reactivity allows to ignore runs that have a zero 
probability of occurring. 

The choice of the verdict, correctness and reactivity notions of a diagnoser thus depends 
on what information the designer of the system desires and on which guarantees are 
demanded. The decisions that are taken also affect the complexity of determining 
the diagnosability and of building the diagnoser. They thus also affect the capacities 
required from the computer that will carry out the diagnosis. Studying as many relevant 
notions as possible and clearly establishing their complexity is thus necessary for an 
appropriate application of diagnosis. 

In Chapter [2j multiple notions of diagnosability were defined for probabilistic sys¬ 
tems based on the ambiguity of specific sets of runs. This allowed us to establish 
diagnosability analysis as a decision problem which is easier to use in proofs and more 
intuitive. However, the end goal of analysing the diagnosability of a system being to 
build a diagnoser, one could also use the following definition: given notions of verdict, 
correctness and reactivity, a system is called diagnosable if there exists a diagnoser of 
the system achieving these features. We will show in Section [l] how to reconcile these 
two approaches: we will associate a diagnoser with each diagnosability notion previously 
defined. Having defined these associated diagnosers will allow us to see how the notions 
of diagnosability translate on an actual run of the system. 

Once the notions of diagnoser have been clearly defined, we must determine how 
the different variants of diagnosability relate to one another. Establishing links between 
multiple notions is a classical part of the semantical analysis of a problem. Through 
this analysis, one can establish that two definitions that are syntactically different, 
are in fact equivalent. In this case, one can choose to use either definition in a proof 
for example. Implications between two notions are also useful. If a system verifies a 
stronger property, we will not have to check for the weaker ones. For diagnosability, it 
allows the system designer to select the strongest available diagnoser without having to 
check every notion. Or at least one of the strongest notions as some of them may be 
incomparable. Therefore, establishing the links between the notions of diagnosability 
defined previously will be the focus of Section [2] 

The last goal of this chapter, presented in Section [3] will be to give, when possible, 
a characterisation of the notions of diagnosability. As we wish to have the simplest 
possible characterisation, an important question is to determine what information is 
needed to characterise diagnosability. For example, can we restrict ourselves to studying 
the structure of the system or do probabilities matter? And if they do, in what way? 
In fact, in the general case, a characterisation relies both on the structure of the system 
and on its probabilities. 
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• The structural part of the characterisation is based on descriptive set theory re¬ 
called in Chapter [2j Section [Tt} We will define a logic generating sets of runs 
belonging to a low level of the Borel hierarchy and associate, when possible, a 
formula with every notion of diagnosability. 

• The probabilistic part then consists in measuring the probability of the set of 
runs identified above. The system is then diagnosable if and only if this measure 
verifies a qualitative requirement. 

Having such a characterisation has multiple advantages. For example, using model¬ 
checking techniques, one could use these characterisations to solve diagnosability as we 
explain in Chapter [5] However this is not necessarily the optimal method as shown in 
Chapter [4] As another example, each Borel set is associated with a level of the Borel hi¬ 
erarchy. The higher this level, the more complicated the set is. This complexity reflects 
a complexity to measure the probability of the set but also a complexity of understand¬ 
ing the meaning behind this set. Having a characterisation with a set belonging to a low 
level of the Borel hierarchy thus makes it easier for the system designer to understand 
the associated diagnosability notion. 

This chapter presents and extends some of the results given in | lBHL14i IBHL16al 
IBHLlfib] . 

1 Diagnoser and diagnosability 

In this section we focus on the synthesis of diagnosers for the notions of diagnosis defined 
in Chapter [2] Recall the definition of diagnosers: 

Definition 3.1. A diagnoser is a function D : X* —>• {?, T, _L} assigning to every finite 
observed sequence a verdict. 

Multiple verdicts can be required for the diagnosers. Intuitively, ? does not provide 
any information, T claims the occurrence of a fault and _L provides information about 
the correctness of the current run. 

Diagnosability as defined in Chapter [2] considers infinite behaviours: either by fo¬ 
cusing on the ambiguity of infinite runs, or by requiring that the probability of a set of 
finite runs converge to 0 when their length diverges to infinity. On the contrary, diag¬ 
nosers are built to react and give an information after a finite number of observations. 
There is therefore no easy direct link between diagnosability and diagnosers. 

Due to its definition, a diagnoser may use infinite memory or more precisely, un¬ 
bounded memory. While infinite memory is not achievable in real systems, unbounded 
memory is. It however raises a question on how to implement this memory and how 
much information has to be kept by the diagnoser. For example, if a diagnoser only 
needs to remember how many observations occurred, it may rely on a counter which, al¬ 
though unbounded, is easy to represent and to modify. When implementing a diagnoser 
(which will be done in Chapter |4j), it is still natural to limit oneself to finite memory. 
We therefore define now the notion of finite-memory diagnosers. 
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Definition 3.2. A finite-memory diagnoser is given by a tuple (M, £ 0 , mo, up, Df m ) 
where: 


• M is a finite set of memory states, 

• mo £ M is the initial memory state, 


• up : M x Yi 0 M is a memory update function, 


Df m : M —> {?, T,_L} is a diagnoser function. 


A finite-memory diagnoser (M, mo, up, Df m ) can be seen as a deterministic au¬ 
tomaton over X 0 where the set of states is M , the initial state is mo and the transi¬ 
tion function is up. Moreover the states of this automaton are labelled by an element 
of {?, T, _L} which is given by the function Dj m . The update up is extended into a 
function up : M X S* —> M defined inductively by up(m, e) = m and up (m,wa) = 
up(up(?n, w), a). The size of a finite-memory diagnoser is given by its number of memory 
states. A finite-memory diagnoser is not a diagnoser as defined in Chapter[2]and recalled 
yet it induces the diagnoser D defined by D(w ) = Df rn (up(mo, w)). 


in Definition 3.1 


Example 3.1. Consider the finite-memory diagnoser (M, {a, b}, mo, up, Df m ) (repre¬ 
sented in Figure 3.1) where 


M = {m, 0 ,m b }, 


• up(m 0 , a) = mo, up(m 0 , b ) = up(m fe , a) = up(m b , b) = m b , 

• -D/to( m 0 ) =? and Df m (m b ) = T. 

It induces a diagnoser D which makes no claim as long as it only observes ‘a ’ and claims 
a fault as soon as a ‘b’ is observed. It then commits to this choice and keeps claiming a 
fault whatever is observed next. 



Figure 3.1: The finite-memory diagnoser of Example 3.1 
a memory state is written below the state. 


The verdict given by Dj m in 


Each following subsection focuses on one notion of diagnosability. They are ordered 
from the easiest to the most difficult definition of diagnoser. Due to the relations that 
will be established in Section [2] we won’t detail every notion of diagnosability here. 
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1.1 FF-diagnosers 

We will start by defining the FF-diagnosers of a pLTS, which is the diagnoser associated 
with FF-diagnosability. Recall that FF-diagnosability requires the probability of the set 
of faulty ambiguous finite runs to converges to 0 (i.e. lim n —^ P(FAmb n ) = 0). These 
diagnosers only provide information about faulty runs, they therefore never raise a _L 
verdict. However they need to raise a T verdict almost surely after a finite delay in 
a faulty run. We can thus restrict their verdict to the set {?, T}. We require from 
diagnosers associated with exact diagnosability notions that they satisfy an additional 
property, commitment , which means that when it claims a fault it will persistently claim 
it in the future. This can be done thanks to the permanence of the fault, i.e. a faulty 
run will remain faulty. 

Definition 3.3. A n FF-diagnoser for a pLTS A is a function D : E* —> {T,?} such 
that: 

commitment For every w <w' £ £*, if D(w) = T then D{w') = T. 

correctness For every w £ £*, if D(w) = T then w is surely faulty. 

reactivity For every p £ minF, P({// £ H | p A p' A D{V{p')) =?}) = 0 where for 
w £ Sq , D(w) = limn-Kjo D(w< n ). 

Let us comment on this definition. The commitment property ensures that if the 
diagnoser outputs T at some point it will always output T. The correctness property 
forbids the diagnoser to claim a fault during the observation of a correct run. This 
reflects that FF-diagnosability is a notion of exact diagnosability. Thus the FF-diagnoser 
is exact too. The limit in the reactivity condition of the above definition is well defined. 
Indeed, note that if T is produced, due to the commitment property, the limit is T. 
Otherwise the diagnoser always outputs ? so that the limit is also a ? verdict. 



a a i 


Figure 3.2: An FF-diagnosable pLTS. 


Example 3.2. For the pLTS of Figure \3.£\ we define the diagnoser D such that D(e) = 
?, for all n £ N, D(a n ) =? and for every word w fL a*, D{w ) = T. This diagnoser is in 
fact the one induced by the finite-memory diagnoser of Example 3.1 Such a diagnoser 
is indeed an FF -diagnoser: 


commitment Once a ‘b’ is observed, every subsequent observation is also a ‘b’, there¬ 
fore once D produces T, it will keep outputting T. 
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correctness If a T has been produced, b was observed. The only transition labelled by 
a ‘b’ is the self-loop on fa which can only be reached by faulty runs. Therefore any 
observed sequence for which D outputs T is surely faulty. 

reactivity Let p be a minimal faulty run. It ends either in fa or in fa. If it ends in 
fi, then with probability 1 a run extending p takes the transition to fa. In fa, it 
will read a ‘b’ in the next step. In other words, with probability 1, a faulty run 
will trigger a ‘b' and thus T is raised by the diagnoser. Due to the commitment 
property, given a faulty run p, the probability that the diagnoser outputs ? infinitely 
often during the observation of a run extending p is thus 0. 

Observe also that this pLTS is indeed FF -diagnosable as for n > 1, we have 
P(FAmb n ) = P ({p £ F n | V(p in ) = a n }) = 

As suggested by the above example, there exists an FF-diagnoser if and only if the 
system is FF-diagnosable. We now prove this formally. 

Proposition 3.1. A pLTS is FF -diagnosable if and only if it admits an FF -diagnoser. 

This proof is done in the following way. Assuming there exists an FF-diagnoser we 
study a family of sets of faulty runs FD„ that corresponds to runs where the fault is 
claimed after n observations. We compute the probability of this set and link this value 
to the probability of FAmb n . This then allow us to show that the probability of FAmb n 
converges to 0, proving thus the FF-diagnosability of the pLTS. Assuming the pLTS is 
FF-diagnosable, we present a diagnoser and then show it is an FF-diagnoser by proving 
the properties one by one. 

Proof. Let A be a pLTS, and assume there exists an FF-diagnoser D for A. For every n £ 
N, we define FD n = {p £ Foo | D(V(p^ n )) = T} the set of faulty runs that are diagnosed 
faulty after n observed events. We will start by showing that lim n —^ P(FD n ) = P(F 00 ). 
As a consequence of the commitment property, the sequence (FD„) ne N is non-decreasing 
and for every faulty run p £ F^, D(V(p )) = lim n ^. 0O D(V(p± n )) =? is equivalent to 

p i UneN( FD «)> i - e - lim ri->oo FD n = UneN( FD «) = {p £ F oo | D(V{p)) /?}. Since D is 
reactive, for every minimal faulty run p £ minF, we have P({// £ 0 | p A p' AD(V(p')) = 
?}) = 0. As every faulty run is prefixed by an unique minimal faulty run, we have 

n{p' € Foq I D(V(p')) =?}) = ]T P (V £ n I p A p' A D{V(p')) =?}) = 0. 

pEminF 


p( U ( FD «)) = G F ~ i ^W)) ^ ? }) 

nSN 

= P(F 0O ) -P({p' £ Foo | D{V{p')) =?}) 

= P( F oo). 


Thus 
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Moreover since D is correct, for every n £ N, FD„ C Sf n . Therefore, for every n £ N, 
P(FAmb n ) = P(F n ) - P(Sf n ) < P(F n ) - P(FD n ) and 

lim P(FAmb n ) < lim P(F n ) — P(FD„) = 0. 

n — 7>oo n —7*-oo 

This shows that A is FF-diagnosable. 

Assume now that A is FF-diagnosable. We define the function D : X* —> {T, ?} by 
D(w) = T if and only if w is a surely faulty observed sequence. Let us check that D 
is an FF-diagnoser. As a surely faulty ambiguous sequence cannot become ambiguous 
again, D fulfils the commitment property. Moreover, since D[w) = T iff w is a surely 
faulty sequence, D is correct. Now, let p be a minimal faulty run. 

P(V £ ft | P ■< p' A D(V{p)) =?}) = lim P {{p' £ FAmb n+ | p \ o \ p < p'}) . 

For every n £ N, we have {// £ FAmb n+ u| o | p < p'} C FAmb n+ ui and, as A is FF- 
diagnosable, lim n _ ) . 0O P(FAmb„) = 0. Therefore P({p' £ D \ p A p 1 A D(V(p')) =?}) = 0 
and D is reactive. □ 

The notion of FF-diagnoser we defined is therefore appropriate for FF-diagnosability. 

1.2 FA-diagnosers 

FA-diagnosability and lA-diagnosability not only consider the diagnosis of faults but also 
of correct runs. Indeed, recall that they require respectively that the probability of the 
set of ambiguous finite runs converges to 0 (he. lim n —> oc P(FAmb n U CAmb n ) = 0) and 
that the probability of infinite ambiguous runs is equal to 0 (he. P(FAmb 00 UCAmb 00 ) = 
0). Contrary to FF-diagnosers, FA- and lA-diagnosers have three possible verdicts: T, 
related to faulty sequences, _L, linked with correctness, and ? when no information can 
be derived from the observation. We consider the partial order -g on these values defined 
by ? -< T and ? -< _L. This order is natural as ? gives less information than the other 
verdicts. Although we consider the detection of correct and faulty runs, note that the 
situation is not symmetric given that the faults are persistent while correct runs may 
become faulty. 

Definition 3.4. An FA-diagnoser for a pLTS A is a function D : £* —> {T, _L, ?} such 
that: 

commitment For every w A w’ £ if D(w ) = T then D(w') = T. 

correctness For every w £ X*, 

• if D(w) = T then w is surely faulty; 

• if D{w) = _L then w is surely correct. 

reactivity P({p £ | D(V(p)) =?}) = 0 where for w £ D(w) = liminf n _ s . 0O D(w< n ). 
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Let us comment on this definition. The commitment property is similar to the one 
of the notion of FF-diagnoser. In particular, there is no commitment for a _L verdict. 
This is natural as a fault could appear later, forcing the diagnoser to change its verdict. 
The correctness property is also similar to the FF-diagnoser for a T verdict. For the 
_L verdict, it is the dual, the diagnoser cannot output _L while observing a faulty run. 
This diagnoser is ‘exact’. The limit in the reactivity requires the partial order. While 
if a T is claimed, the limit will be T, a 1 verdict can be followed by a ? verdict. Due 
to the correctness property, the limit is equal to _L if the observed sequence is, after a 
finite number of observations, always surely correct. 

c 

a 


a at, 

Figure 3.3: An FA-diagnosable pLTS. 




Example 3.3. For the pLTS of Figure 3.3 we define the diagnoser D such that D{e ) = 
_L and given an observed sequence w 6 £*, then D(wb ) = T, D(wc) = _L and D(wa ) = 
D{w). Remarking that once a b or a c is observed, the other cannot appear any more, 
this diagnoser is induced by the finite-memory diagnoser of Figure 3.f Such a diagnoser 
is indeed an FA -diagnoser: 

commitment Once a ‘b’ is observed, every subsequent observation is also a ‘b ’, there¬ 
fore once D produces T, it will keep outputting T. 


correctness If a T has been produced, ‘b ’ was observed previously. The only transition 
labelled by a ‘b’ is the self-loop on // which can only be reached by faulty runs. 
Therefore any observed sequence for which D outputs a T is surely faulty. Simi¬ 
larly, when D outputs _L it means that a ‘c’ was observed previously and such an 
observation can only be made in q\ which is a correct state from which no fault 
can ever be triggered. Thus the observed sequence is surely correct. 

reactivity The set of infinite observed sequences w for which D{w) =? is restricted 
to {a^}. There exist exactly two rims corresponding to this observed sequence 
Pi = Qo u (Qi a ) u an d P 2 = Qo^{fi a ) UJ - Moreover, the probability of each of these two 
runs is 0 as firing ‘a’ in q\ or in f\ only has probability 1/2. Thus with probability 
1, a ‘b’ or a ‘c’ will be observed, ensuring the reactivity of D. 

Observe also that this pLTS is indeed FA-diagnosable as for every n > 1, we have 
FAmb„ = {qou(qia) n qi,q 0 f(fia) n fi,q 0 f(fia) n ~ 1 fiaf 2 }. Thus P(FAmb„) = . 

We now want to establish the link between the existence of an FA-diagnoser and 
FA-diagnosability. However, there is no equivalence in the general case. Indeed, let us 








Diagnoser and diagnosability 


63 


a 



Figure 3.4: The finite-memory diagnoser of Example |3.3[ 


consider the pLTS of Figure 3.5 The probability of the set of ambiguous runs of length 
n > 1 starting by u\ is the probability to have read a ‘6’ on the n — l’th observation. 
This is equal to ^j. Moreover, runs initially starting by U 2 will almost surely trigger a 
‘c’, removing the ambiguity. Thus it is FA-diagnosable. However, a run starting with u\ 
will almost surely trigger infinitely many l b 7 s. Because of the correctness property, we 
have for every diagnoser D , P({p G fl \ D(V(p)) =?}) > P({/9 G Fl | qoUiqi A p }) = 
Thus, D is not reactive. 



Figure 3.5: An infinite FApLTS which does not accept any FA-diagnoser. 


Fortunately, the equivalence holds when restricted to finite pLTS. We postpone the 
proof of the following proposition to Chapter [d] (more precisely to Proposition 4.14 
page 126) which focuses on finite pLTS. 


Proposition 3.2. A finite pLTS A is FA-diagnosable if and only if it admits an FA- 
diagnoser. 


As a conclusion, the definition of FA-diagnosers is similar to the one of FF-diagnosers, 
but with additional requirements to deal with the correct ambiguous runs. This fits the 
definition of FA-diagnosability which becomes equivalent to the one of FF-diagnosability 
when the set of correct ambiguous runs can be neglected. However, due to the complex¬ 
ity created by the fact that being correct is not a permanent status of runs (contrary 
to being faulty), the link between existence of an FA-diagnoser and FA-diagnosability 
cannot be established in the general case. 
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1.3 lA-diagnosers 

The problem in defining lA-diagnosers is that lA-diagnosability defines ambiguity for 
infinite runs while a diagnoser must give its verdict after a finite observation. As a 
consequence, the information an lA-diagnoser gives after a finite run is weaker than the 
information an FA-diagnoser has to give. 

Definition 3.5. An lA-diagnoser for A is a function D : X* —>• {T, _L, ?} such that: 
commitment For every w A w' G £*, if D(w) = T then D{w') = T. 
correctness For every w £ £* 

• if D(w) = T, then w is surely faulty; 

• if D(w) = _L, letting |D(u;)|j_ = |{0 < n < |u;| | D(w< n ) = _L}|, then for 
every signalling run p such that V(p) = w, Pi\d(w)\± correct. 

reactivity P({p £ ft \ D(V(p)) =?}) = 0 where for any observed sequence w € Tff, 
D(w) = lim sup n ^. oc D(w< n ). 

Let us comment on the definition. Commitment is once again focused only on the T 
verdict. Correctness is usual for T but quite different for _L. Indeed, the correctness of 
FA-diagnosers requires that a _L verdict means the observed sequence is surely correct. 
The interpretation of D{w ) = _L for lA-diagnoser is that the diagnoser ensures that any 
signalling subrun of length |-D(ie)|j_ < \w\ of a signalling run for w is correct. Of course 
it may deduce this information from the last |rc| — |D(u;)|_|_ observations. This does not 
reveal if the current run is correct or not. However, if the diagnoser outputs _L infinitely 
often along an observed sequence w, lim,,^,^ \D(iv< n )\± = oo. Therefore the infinite 
observed sequence w is surely correct. The reactivity condition uses a limit superior as 
we only need _L to be claimed infinitely often but not necessarily without ? in between. 



Figure 3.6: An lA-diagnosable pLTS. 


Example 3.4. For the pLTS of Figure 3.6 we define the diagnoser D by: for any 
observed sequence w if there exists an observed sequence w' such that w = w'c then 
D{w ) = T, if w 6 {w'aa, w'ab}, D(w ) = _L else D{w) =?. This diagnoser is induced 
by the finite-memory diagnoser represented in Figure [X?| Such a diagnoser is indeed an 
IA -diagnoser: 


commitment Once a ‘c’ is observed, only a ‘c’ can appear, thus the diagnoser keep the 
T verdict. 
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correctness Observing a ‘c ’ can only be made for faulty run, thus D correctly raises 
a T. For the correct runs the idea is the following. After observing any sequence 
waa or wab, with w E {a,b}*, the diagnoser knows a posteriori that one step 
before, that is after the observation of wa, the run was necessarily correct. Indeed, 
observing the suffix aa is not possible after a fault, yet wba is not surely correct. 
After a run p with such an observation we thus know that p^yp( p )\-\ is correct 
and \D(w)\± is at most equal to \P(p)\ — 2 as no _L can be claimed after 0 or 1 
observation. Thus D is correct. 

reactivity With probability 1 a faulty run will trigger a ‘c’ (raising a T verdict) and a 
correct run will trigger infinitely many ‘a’s (raising infinitely many _L verdicts). 

Moreover this pLTS is I A -diagnosable as CAmboo U FAmboo = V 1 ({a,6}*6 CJ ) which 
has probability 0 as in every state there is a positive probability to trigger an action 
whose observation is different than b. 



Figure 3.7: The finite-memory diagnoser of Example 3.3 


We now aim at establishing the link between lA-diagnosability and the existence 
of an lA-diagnoser. Since lA-diagnosability gives an information about infinite runs, 
we need a way to translate it to finite runs in order to establish the link with the 
diagnoser. Hence, we first introduce a lemma linking the sets FAmb n and FAmboo for 
n E N. This lemma will be reused in the next section when establishing the link between 
FF-diagnosability and I F-diagnosability. 

Lemma 3.1. Let A be a pLTS. Then lim n _>.oo IP(FAmboo \ FAmb n ) = 0. Moreover, if A 
is finitely branching, then lim n _ 5 . 00 P(FAmb n \ FAmboo) = 0. 

The main difficulty of the following proof is in the second point. There, using the 
finitely-branching assumption and invoking Konig’s lemma, we show that if the prefixes 
of a finite run are never surely faulty, then we can build an infinite correct run with the 
same observed sequence. 
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Proof. Observe that £1 admits the following partitions f l = FAmboo l+J Coo O Sfoo and for 
all n £ N, £l = FAmb n l±J C„ 1+) Sf n . Thus, for all n E N, 


FAmboo \ FAmb„ = (C„ l+J Sf„) 0 FAmboo 

= (C n l+J Sf n ) \ (Coo ttl Sfoo) C (C n \ Coo) O (Sfn \ Sfoo). 

Since for all n, Sf n C Sfoo, one gets: 

FAmboo \ FAmbn f-n \ f-oo • 

{C n }neN is a non-increasing family of sets and we claim that Coo 
an infinite run p is correct if and only if f does not occur in it 
all its signalling subruns are correct. Thus, lim„_>.oo P(C n \ Coo) 
lim n _>oo P( FAmboo \ FAmb n ) = 0 . 

For the other direction, using again the two partitions we obtain: 

FAmb n \ FAmboo = (Coo W Sfoo) Cl FAmbn 

= (Coo W Sfoo) \ (Cn l+J Sf n ) C (Coo \ C n ) W (Sfoo \ Sf n ). 

Since for all n, Coo ^ C n , one gets: 

FAmb.„ \ FAmboo C Sfoo \ Sf„ 

Let us show that, under the assumption that A is finitely branching, Sfoo UneN Sfn- 
Let p fi UneN Sfn- We build a tree as follows: 

• Nodes at level n correspond to the correct signalling runs whose observed sequence 
is Vipin); 

• The node at level n +1 associated with p' is a child of the node at level n associated 
with p" if p” < p'. 

Since p ^ U n eN Sfn, f° r all n G N, there exists a correct run with observed sequence 
V{pin ), so that the above-dehned tree is infinite. Since the pLTS is finitely branching 
and convergent, the tree is also finitely branching. By Konig’s lemma, it must contain 
an infinite branch, thus there exists an infinite correct run whose observed sequence is 
V(p). As a consequence p is not surely faulty: p Sf^. This establishes that Sfoo ^ 
UneN Sfn- Thus liirin—>oo P(Sfoo \ Sf„) = 0 implying lim„^oo P(FAmb n \ FAmboo) = 0 
which concludes the proof. □ 

We can now establish the following proposition. 

Proposition 3.3. A finitely-branching pLTS A is IA -diagnosable if and only if it admits 
an IA -diagnoser. 


= rineN C «- Indeed 
i.e. if and only if 
= 0 which implies 
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Proof. Assume first that there exists an lA-diagnoser D for A, and let p be an infinite 
run. By reactivity, almost surely D(V(p)) £ {T,_L}. If D(V(p)) = T then there 
exists some n such that D(V{p^ n )) = T. By correctness, p\ n is surely faulty and thus 
p is surely faulty. If D(fP(p)) = _L, we claim that p is surely correct. Observe that 
the diagnoser infinitely often outputs _L, so by correctness, for all n, V(p± n ) is surely 
correct and thus in particular p^ n is correct. Assume there exists an infinite faulty run 
p' with V(p') = V(p). There exists a n such that for all m > n, p'^ n is faulty. Thus by 
correctness there can be no more than n T verdicts for V(p) contradicting the fact that 
D(V(p)) = _L. Thus with probability 1, an infinite run is unambiguous. 

Conversely, assume that A is lA-diagnosable. Given an observed sequence w, we 
denote by N w the largest integer such that Cyl('P~ 1 (u>)) n Fn w = 0, *-e. the largest 
integer such that every run with observation w was correct after N w observations. We 
define the diagnoser D such that D(e) =? and for every observed sequence w and 
observation o £ £ 0 , if wa is surely faulty, then D(wa) = T, if N wa > N(w), then 
D(wa) = J_ else D(wa ) =?. 

• Commitment is direct from the definition of D. 

• Correctness is achieved as T is raised for surely faulty runs and _L is raised 

when N w (for appropriate observed sequence w) increased, thus for all w £ £*, 
\D(w)\± < N w which implies that for a run p £ PnD(w)\± is correct. 


Reactivity, however is a bit more complicated, we need the result of Lemma 3.1 
Let p 0 FAmboo U CAmboo. 

— If p is correct, then suppose that there exists no such that for all n £ N, 
there exists a run pf £: Fn 0 with P(p'[ n ) = 'P(pin). Then using Konig’s 

there exists 
, and raise a 


lemma and a construction similar to the one of Lemma |3.1 
Pf £ ' n o 


F no such that V(p) = V{pf) which would mean p £ CAmh 
contradiction. Thus, as for each uq there exists n £ N such that A T v( Pin ) > no 


-^P(p) = lim n —>oo N' 


'P(p^n) 


= oo. 


As every time this value increases, _L is 


produced by D 1 D outputs infinitely many _L, thus D(V(p)) = _L. 

0 P(FAmb r 


— If p is faulty, according to Lemma 3.1 lirn 


\ FAmboo) = 0. 

Therefore, with probability one, there exists no £ N such that p 0 FAmb„ for 
n > no- Let ni such that p £ F n , and n 2 = max(no,ni), then by definition 
of D,D(V(p in2 )) = D(V(p)) = T. 


Therefore D is reactive. 


Thus, D is an lA-diagnoser. □ 

I A-diagnosers are thus appropriately associated with lA-diagnosability. They manage 
to give information after a finite amount of time about infinite ambiguity. We start 
to see a complexity hierarchy between the various exact diagnosability notions. FF- 
diagnosability appears as the simplest notion as its equivalence with the existence of 
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an FF-diagnoser was established in the general case. Second comes lA-diagnosability 
for which we needed to restrict our framework to finitely-branching pLTS. Finally, the 
most difficult notion is FA-diagnosability for which the equivalence only holds for finite 
pLTS. This hierarchy is however only an intuition for now as it could be the result of 
an inappropriate choice of diagnosers definitions. 


1.4 eFF-diagnosers 


We now define the eFF-diagnosers corresponding to eFF-diagnosability for e > 0. Recall 
that eFF-diagnosability requires that, after any minimal faulty run p, the probability of 
the set of faulty runs extending p whose observed sequence has a correctness proportion 
(CorP, defined page 46) above e converges to 0 ( i.e . lim n —^ P(FAmb^) = oQ Given 
£>0, eFF-diagnosers are similar to FF-diagnosers in the sense that both only consider 
faulty runs and thus never output a _L verdict that would give information about correct 
runs. However, eFF-diagnoser may make errors: when they give a T verdict, then the 
probability that the claim is wrong is at most e. 


Definition 3.6. Let e > 0. An eFF-diagnoser for A is a function D 
siLch that: 

correctness For every w E £*, if D(w) = T then CorP(u)) < e; 
reactivity limsup^^P({p E Ffl SR„ | D(V(p)) =?}) = 0. 


{T,?} 


Let us comment on this definition. This diagnoser is no longer exact. Given an 
observed sequence w, it is allowed to output T, if the probability of error is below e as 
shown by the requirement CorP(u>) < e. There is no longer a notion of commitment, 
allowing the diagnoser to go back from a T verdict to a ? verdict. This absence of 
commitment is one of the differences between the definition of eFF-diagnosers and the 
one of monitors for distinguishabihty of hidden Markov chains |SZF111 IKS16| . See the 


discussion after Lemma |4.3[ page 108 for more details about the links between monitor¬ 
ing and diagnosability. Finally, the reactivity condition as defined here is different from 
what was done for the other diagnosers. This choice was made in order to be closer to 
the corresponding diagnosability notion as approximate notions of diagnosis are harder 
to handle than exact ones. 

One could introduce a uniform variant of this definition that would correspond to 
uniform eFF-diagnosability. However, this definition would not follow the same structure 
as the other ones as a uniform reactivity would have to be defined on individual runs 
instead of on the global conditions we used here. 

Example 3.5. Let us observe the pLTS on the left of Figure \3.8\ We define the diag¬ 
noser D such that given an observed sequence w E T,*, then D(w ) = T iff w contains 
at least as many ‘b’ than ‘a’, else D(w) =?. Clearly, this diagnoser does not satisfy any 


1 This definition is written slightly differently from the one of Definition 2.9 page |48| Yet, one 


can quickly see that they are equivalent. However, we cannot easily express uniformity with a similar 
definition. 
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Figure 3.8: Left: an l/2FF-diagnosable pLTS. Right: a pLTS which is not eFF- 
diagnosable for any e > 0. 


commitment property, but it is not required for eFF-diagnosers. Moreover, this diagnoser 
cannot be translated into a finite-memory diagnoser as we need to keep the difference 
between the number of occurrences of ‘a ’ and ‘b ’. We give a visual representation, using 
an infinite number of states, of this diagnoser in Figure 3.9. Such a diagnoser is a 
1/2FF -diagnoser: 


correctness Given an observed sequence w, if D(w) = T, then CorP(u;) = 
As by definition of D, |u;| a < \w\b, CorP(?n) < 1/2, thus D is correct. 


reactivity Let a > 0, as faulty runs have a probability 3/4 to raise a ‘b’ at each step, 
according to the weak law of large number there exists no E N such that for all 
n > no, P({p E F n \ \V(p)\ a > [P{p)\b\) < a ■ Let p be a faidty run such that 
D (P(p)) = ? - Thus by definition of D, p E {p' E F n | \V{p')\ a > \V(p')\b}- 
Therefore, for n > no, P({p E F n SR n | D(V(p)) =?}) < P({p E F n | |u;| a > 
|u;|&}) < a. Thus D is reactive. 

1 /2 

Observe also that this pLTS is indeed 1/2FF -diagnosable as for n > l,P(FAmbn j = 
P({p E F n | \V(pin)\ a > ^{pi^lb}) which converges to 0 according to the weak law of 
large numbers. 



Figure 3.9: An automaton representing the diagnoser of Example |3.5[ 


The diagnoser used in Example |3.5| requires unbounded memory. In fact, there is 


no finite-memory l/2FF-diagnoser for the pLTS on the left of Figure 3.8 


Proposition 3.4. There exists a 1 /2FF -diagnosable pLTS that admits no finite-memory 
1/2FF -diagnoser. 
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and assume there 

exists a l/2FF-diagnoser with m states. After any sequence o n for n > 1, it cannot 
claim a fault as CorP(a n ) = jjtrpy > \. So there exist 1 < i < j <m + 1 such that the 
diagnoser is in the same memory state after observing a* and . 

Consider the faulty run p = qofqf (aqf) 1 . Due to the reactivity requirement, there must 
be a run pp' for which the diagnoser claims a fault. Thus for all n. the diagnoser 
also claims a fault after p n = but lim, woo CorP (V(p n )) = 1, which 

contradicts the correctness requirement. □ 


Proof. Consider the l/2FF-diagnosable pLTS on the left of Figure 3.8 


We now establish the link between eFF-diagnoser and eFF-diagnosability. 

Proposition 3.5. Let e > 0. A pLTS A is eFF -diagnosable if and only if it admits an 
eFF -diagnoser. 

This proof has more computations than the previous ones due to the approximate 
notion of correctness, however the main ideas are similar: from an eFF-diagnoser we 
relate FAmb^ to {p' G F n | D(V(p')) =?} in order to show that the probability of 
FAmb^ converges to 0. In the other direction, assuming A is eFF-diagnosable, we build 
an eFF-diagnoser that is reactive for the runs that do not belong to FAmb^ infinitely 
often. 


Proof. Let A be a pLTS and e > 0. Assume that there exists an eFF-diagnoser D for 
A. Let p be a minimal faulty run and a > 0. Since D is reactive, there exists n PiQ G N 
such that for all n > n PtCt , 

P({p' <E F n SR n I D(V(p')) =?}) < a • P(p) . 


Thus for all n > n Pt0 ,: 

P ({P € SR n+ | p | o | D(V(p')) =? A p A p’}) < P {{(f G F n SR n+ | p | o | D(V(p')) =?}) 

< a ■ P(p) . 


Since D is correct, Cyl(p) C FAmbf l+ | p | o C Cyl({p' G SR n+ | p | o | D{V(p')) =? A pP p'}). 
Thus P(Cyl(p) n FAmb^ + ^| o ) < a ■ P(p) . This establishes that A is eFF-diagnosable. 
Conversely assume that A is eFF-diagnosable. Let D be the diagnoser defined by: for 
all w G £*, D(w ) = T iff CorP(u;) < e. Such an eFF-diagnoser is correct by definition. 
Let a > 0. Since (F n ) ne N is a non-decreasing sequence converging to Foo, there exists 
no G N such that for all n > no, P(F n \ F no ) < a/2. By eFF-diagnosability of A, for all 
p G Ufc<n () minFfc, there exists n p such that for all n > n p 

P(Cyl(p)nFAmb e „ +|p|o )<|-P(p). 
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Define n max = minFn Then for n > n 0 + n max we have 

P(FAmb^) < P(FAmbf 1 n F no ) + P(FAmb* \ F„ 0 ) 

< E IP(Cyl( /0 )nFAmb £ J + P(F n \F no ) 

PeUfe<„ 0 minF fc 

E CK / N Qi 

2 - p (p) + 2 - 

PeUfe<, 10 minF fc 

So we have established that lim n ^. 00 P(FAmb^) = 0. 

By definition of D. FAmb^ = {p E F n | D(V(p)) =?}. Thus D is reactive. D 

For every e > 0 we have thus an appropriate notion of eFF-diagnoser associated 
with eFF-diagnosability. As a pLTS is AFF-diagnosable if it is eFF-diagnosable for 
every e > 0, we directly obtain the following corollary. 

Corollary 3.1. A finite pLTS A is AFF-diagnosable if and only if for all e > 0 it admits 
an eFF-diagnoser. 

In other words, when a pLTS is AFF-diagnosable, the designer can choose the accu¬ 
racy they want for the diagnoser. 


2 Relationships between diagnosability notions 


In this section, we establish the links between the multiple notions of diagnosability 
defined in Section |1.5[ We gave in Section [l] diagnosers associated with the various 
notions of diagnosability. Thus, intuitively requiring a stronger version of one feature 
(verdict, correctness or reactivity) defines a diagnoser that gives more information and 
thus which is less likely to exist. For example, the difference between FF-diagnosability 
and FA-diagnosability is that FA-diagnosability must identify faulty and correct runs, 
while FF-diagnosability only cares about faulty runs. Thus FA-diagnosability implies 
FF-diagnosability, which can be formally proven immediately since for all n, FAmb n C 
FAmb n l±J CAmb n . The two notions being entirely distinct as FF-diagnosability does not 
imply FA-diagnosability as shown by the pLTS of Figure [3. 10| there is a single ambiguous 
observed sequence for every n £ N, a”, this sequence can be observed with a probability 
1/2 of correct runs and by a probability l/2 n of faulty runs, i.e. Vn E N,P(CAmb n ) = 
1/2 and P(FAmb n ) = l/2 n therefore it is FF-diagnosable without being FA-diagnosable. 
Similarly, as FAmboo C FAmboo l±J CAmboo, lA-diagnosability implies I F-diagnosability 
The converse is not true however as the pLTS of Figure 3. 1C ! is I F-diagnosable while it 
is not lA-diagnosable. 

An interesting case is the link between I F-diagnosability and FF-diagnosability. In¬ 
tuitively, as for I F-diagnosability we can observe the infinite run before giving a verdict 


2 Note that rninF n is finite due to A being finitely branching and convergent. 


d This pLTS was already displayed in Figure 3.2 
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a a 5 

Figure 3.10: An FF/IF-diagnosable pLTS which is not FA/IA-diagnosable. 


while FF-diagnosability only allows for a finite observation, FF-diagnosability implies 
IF-diagnosability. This is indeed true. Using the first direction of the Lemma 3.1 
we know that lim n _ >.00 P(FAmb 0O \ FAmb n ) = 0. Thus if a pLTS is FF-diagnosable, 
lim n _>-00 P( FAmb n ) = 0 , therefore lim n _>oo P(FAmboo) < lim n ^. 0 oI I> (FAmb 0O \FAmb n ) + 
lim n — P(FAmb„) = 0. Hence the pLTS is IF-diagnosable. Moreover, the impli¬ 
cation is strict. Observe the pLTS of Figure 3.11, it is IF-diagnosable (and even 


lA-diagnosable in fact) as every correct run ends with an infinity of b which can¬ 
not be observed with a faulty run and the only faulty run only triggers the obser¬ 
vation a, thus FAmboo tt) CAmboo = 0. However, it is not FF-diagnosable as for all 
n £ N, P(FAmb n ) = 1/2. Indeed, for n G N, there exists a transition from q$ to a state 
q n 1 labelled by an a and with probability l/ 2 n+1 , the one correct run p of observable 
lengthn starting by this transition has observation V(p) = a n , thus the only faulty run of 
observable length n, qof (/ia) n /i is ambiguous. Moreover this faulty run has probability 
1/2. Although we have a strict implication, the pLTS used to prove the strictness has 
an infinite branching. This is in fact necessary: given a finitely-branching convergent 
pLTS, I F-diagnosability is equivalent to FF-diagnosability. Given a pLTS A, the impli¬ 


cation from I F-diagnosability to FF-diagnosability is shown using once again Lemma 3.1 
which, assuming finite branching, says that limn—>oo P(FAmb n \ FAmboo) = 0 . Thus if A 
is IF-diagnosable, lim„_^ P(FAmbn) = lim n _ >00 P(FAmb„ \ FAmboo) + P(FAmb 00 ) = 0. 
Thus A is FF-diagnosable. 



Figure 3.11: An infinitely-branching pLTS that is lA-diagnosable but not FF- 
diagnosable. 


Although the relationship between lA-diagnosability and FA-diagnosability is the 
same as the one between I F-diagnosability and FF-diagnosability, the same link cannot 
be established. Indeed, even for finite pLTS, lA-diagnosability does not imply FA- 
diagnosability. Let us observe the pLTS of Figure 3.12 It is lA-diagnosable, indeed, 


every infinite correct run have observed sequence a u while the observed sequence of 
every infinite faulty run is of the form a n 6 aJ for n > 0 , thus CAmboo G FAmboo = 0 - 
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Consider however the infinite correct run p = qouqi(aqi) u . It has probability ^. and all 
its finite signalling subruns are ambiguous since their observed sequence is a n , for some 
n 6 N which is the observed sequence of the faulty signalling run qou(q 2 a) n ~ 1 q 2 t fiaf 2 - 
Thus for all n > 1, P(CAmb n ) > so that this pLTS is not FA-diagnosable. 



a a b b 


Figure 3.12: An lA-diagnosable pLTS which is not FA-diagnosable. 


The next theorem summarises the connections between the different diagnosability 
notions. 


Theorem 3.1. The diagnosability notions for pLTS are related according to the diagram 
below, where arrows represent implications. All implications, except the one from IF- 
diagnosability to FF -diagnosability and the one from FF -diagnosability to uniform FF- 
di a gn os ability, hold for arbitrary infinite-state pLTS. The latter implications holds for 
finitely-branching pLTS and finite pLTS respectively. Implications that are not depicted 
do not hold, already in the case of finite-state pLTS. 


uniformly 
FF -diagnosable • 


uniformly 
AFF -diagnosable 

for all e > 0 


miiformly 
eFF -diagnosable 


for finite pLTS/ 


FA-diagnosable —► FF-diagnosable • 


for finitely / 
branching pLTS\ 


for all £ > 0 
AFF -diagnosable -► eFF-diagnosable 


-► OFF -diagnosable 


I A -diagnosable —► IF -diagnosable 


Let us first describe this diagram. Omitting OFF-diagnosability which is equivalent 
to FF-diagnosability, the first two columns correspond to exact diagnosis (thus diag¬ 
nosis where the diagnoser cannot claim a fault if there is a probability that the fault 
did not occur) while the last two rows correspond to approximate diagnosis. The lower 
row contains the notions of diagnosis considering infinite runs, the middle row the ones 
considering finite runs and the upper rows the ones considering finite runs and requiring 
uniformity on the speed of reactivity of the diagnoser. From any notion of diagnosabil¬ 
ity, the notion above it, if there is one, has a more restrictive reactivity and the one 
on its left requires a better correctness or has a verdict extended to more elements. 
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FF-diagnosability plays a central role in the diagram as the notion of uniformity and 
approximate diagnosis was built from this notion. This notion was chosen as it is the 
traditional notion of diagnosis |TT05] . moreover there is no clear intuition of what 
approximation and uniformity means for infinite runs. 


Proof. We prove all implications that were not already stated in the beginning of this 
section. Most are pretty straightforward. For all e > 0, the implications from AFF- to 
eFF, uniform AFF- to uniform eFF- and uniform eFF- to eFF- are direct by definitions. 
The implications from uniform AFF- to AFF-, uniform FF- to FF-, FF- to AFF- and uni¬ 
form FF- to uniform AFF- comes partially from definitions and from application of other 
implications (mostly the equivalence between FF- and OFF- proven below). The most 
complicated implications are (1) the equivalence between FF- and OFF-diagnosis which 
is a careful inspection of sets of runs, taking account the possibly infinite branching, ( 2 ) 
the implication from FA- to lA-diagnosis which is inspired by Lemma 3.1 and (3) the 
implication from FF- to uniform FF-diagnosability for finite pLTS which requires the 
characterisation of FF-diagnosability for finite pLTS which will be established later and 
is thus postponed (See Proposition 4.3 page 95). 


FF 4^ OFF. 

Let A be a OFF-diagnosable pLTS and e > 0. Since (F n ) ng N is a non-decreasing 
sequence converging to Fqq, there exists no E N such that for all n > no,P(F n \ 
F»io) < §• By 0FF-diagnosability of A. for all p E Ufc<n„ m ' n Ffc, there exists n p 
such that for all n > n p 

P(Cyl(p) n FAmb n+ | p | o ) < | • P(p). 

Notice that, because the pLTS may be infinitely branching, the set U fc<ri() minF^ 
may be infinite. We therefore define n m ax such that P({p E U fc<r)() min| n p > 

4 " Thus, only a small portion of runs p m l_J ^<^^0 rninF^, have n p > T^max' 

Then for n > no + n max we have 


P(FAmb n ) < P(FAmb n \ F no ) + P(FAmb n n F no ) 

< P(FAmb n \F no )+P({pE J min F k | Tip ^ Tl'fYiax}') 

k<no 

+ P {{p' € FAmb n | 3p E |J minF fc , p < p', n p < n max }) 

k<n 0 

< h + 1 + e U minFfc I n p - ^ e • 

k<no 

Let A be a FF-diagnosable pLTS. Consider p E minF and a > 0. There exists 
no € N such that for all n > no, P(FAmb n ) < a ■ P (p). Thus for all n > no: 


P(Cyl(p) n FAmb n+ | p | o ) < P(FAmb n+ | p |J < a • P (p). 







Relationships between diagnosability notions 


75 


FA => IA. 

For all n E N, define CAmb ni00 the set of correct ambiguous runs that admit an 
observationally equivalent run which is faulty before its n th observable event. 
Observe that the sequence of sets {CAmb ni00 } ng N is non-decreasing and that 

CAmboo 


— UneN CAmb ni00 . 


Moreover, by definition, CAmb ra;00 C CAmb n . As¬ 


sume that limsup n _ ) . 0O P(FAmb r! , l±J CAmb„) = 0. By Lemma 3.1 P(FAmboo) = 0. 
For all e > 0, there exists n± E N such that for all n > n\, P(CAmb n ) < e and 
thus P(CAmb n)00 ) < e. On the other hand, there exists ri 2 E N such that for all 
n > n 2 , P(CAmb 00 ) — P(CAmb„ j00 ) < e. Combining these two inequalities for 
n = max(ni,n 2 ), one obtains P(CAmboo) < 2e. As e is arbitrary, P(CAmboo) = 0. 


We now provide counter-examples for the implications that do not hold and which 
were not already developed at the beginning of the section. The most interesting exam¬ 
ple is the one establishing the difference between FA-diagnosis to uniform eFF-diagnosis 
as it requires an infinite pLTS. 


uniform AFF =£> IF. 


Consider the pLTS depicted on the left of Figure 3.8 All infinite faulty runs 


are ambiguous, and the probability of faulty runs is g, thus this pLTS is not 
IF-diagnosable. Fix some e > 0 and 0 < a < 1. There are two minimal faulty 
runs p a = qoiqfdqf and p = qoiqfbqj. Consider first p a and let p be the random 
variable of a signalling run of length n that extends p a . One can express the 
correctness proportion of p in terms of the number of a’s in its observed sequence, 
written \p\ a : 


CorP(p) = 


(3 ) H a( l ) |pHp|a + (l)|pL(| ) |p|-|p| 0 


Simplifying this expression, we obtain: CorP(p) = 1 +3 | p |_ 2 |p| a • Now, by the strong 
law of large numbers, for any ?y > 0, there exists n v such that for every n > 
P(|4|p| a — |p|| > rj) < a. So with probability at least 1 — a, the correctness 
proportion of p is bounded by - li+\ P \ ■ F° r a sufficiently large rj, this value is 

smaller than s, so that P(CorP(p) < e) > 1 — a. 


A similar reasoning applies to pb, and one can then take the maximum of the two 
integers n v to prove that the pLTS is uniformly AFF-diagnosable. 


b 


l 
’ 4 


a 


3 
’ 4 


a 


3 _ 

- 16 



6,1 


Figure 3.13: An AFF-diagnosable pLTS which is not uniformly eFF-diagnosable for 
e < 3/4. 
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AFF=?> uniform eFF. 

Consider the pLTS of Figure 


3.13 


Fix some 0 < e < |, 0 < a: < 1 and n a . 
Consider the minimal faulty run p = qouqi(aqi) na+1 fqfbqf. Let p' be the sig¬ 
nalling run of length 2 n a + 2 such that p A p'. V(p') = a n °‘ +1 b n °‘ +1 . Thus, 
CorP(iP {(/)) > §. So 


P(Cyl (p) C FAmb| nct+2 ) = P(p) > a 


Thus the pLTS is not uniformly eFF-diagnosable. 

Let p be a minimal faulty run. Then V(p) = a n °b for some uq. For all n, let p n 
be the single the single signalling run of observable length \p\ + n that extends p. 
It fulfils V(p n ) = a n °b n+1 and P(p n ) = P(p). The single correct signalling run p' n 
with T(p' n ) = V(p n ) fulfils P (p' n ) = 2 - 4 4 ) +n+i ■ Thus lim^oo CorP (V(p n )) = 0. So 
the pLTS is eFF-diagnosable for all e > 0 and thus AFF-diagnosable. 

FA =&■ uniform eFF when considering infinite pLTS. 

Let us consider the pLTS of Figure |3.14| It is FA-diagnosable as almost surely 
a faulty (resp. correct) run contains a b (resp. c) after a finite number of steps 
that cannot be mimicked by a correct (resp. faulty) run. We claim that it is not 
uniformly eFF-diagnosable for all e such that 0 < e < Note that for all n G N, 
CorP(a n ) > ■ Tix some 0 < a < 1 and n a € N. Consider the minimal faulty 

run p = qouf\af 2 ■ ■ ■ o,f na ff' na . The shortest extension of p that is not ambiguous 
(he. contains a b) contains n a + 1 observable events more than p does. Therefore, 
P({p' € FAmb^ +Ho | p A p'}) = P(p) > a • P(p). 



c b 


Figure 3.14: An infinite FA-diagnosable pLTS that is not uniformly eFF-diagnosable. 


uniform eFF =&■ AFF. 


Consider the pLTS of Figure 3.15 There is a single signalling minimal faulty run 


qofqfaqf. Any observed sequence of length at least 1 is ambiguous and corresponds 
with equal probability to a signalling correct or a faulty run. Consequently it is 
not AFF-diagnosable, yet it is uniformly eFF-diagnosable for £ = \- 


This concludes the proof. 


□ 
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Figure 3.15: A uniform ^FF-diagnosable pLTS, that is not A FF-diagnos able. 


3 Characterisation of diagnosability 


Our goal in this section is to establish “simple” characterisations of the diagnosability 
notions for a pLTS. More precisely, we aim at studying whether there exists a Borel 
set B G B that only depends on the underlying LTS such that almost surely a random 
run belongs to B if and only if the pLTS is diagnosable. Allowing B to depend on 
the probabilities of the pLTS would give more expressivity, but strongly increases the 
complexity of the information contained in the set which goes against the goal of a 
“simple” characterisation. Similarly, as much as possible, we look for a set B belonging 
to a low level of the Borel hierarchy. 

Approximate notions of diagnosability heavily depend on the probabilities of the 
transitions. This can be seen on the two pLTS of Figure 3.8, page |69| for example: 


while the one on the left is eFF-diagnosable for every £ > 0, the one on the right, 
which is obtained simply by swapping two probabilities, is not eFF-diagnosable system 
for all e > 0. We thus focus here on FA/FF/IA/IF-diagnosis. For these notions, the 
characterising Borel sets cannot be obtained directly from the definitions and require 
some machinery. Indeed, the notions of FF- and FA-diagnosability are expressed by 
the limit of the probability of a family of open sets and the notions of IF- and IA- 
diagnosability are expressed by a set which is not a priori a Borel set. 

We will specify these Borel sets using a logic called pathL. Logics are tools that are 
efficient at giving specifications for a system and, thanks to model checking, at deciding 
if these specifications are satisfied by the system. The complexity of model checking a 
logic depends on the kind of systems considered (finite, probabilistic,...) and on the 
power of expressivity of the logic. This makes thus yet another argument for requiring 
simple characterisations of the diagnosability notions. 

Then, in Subsection 3.2 


In this section, we define the logic pathL in Subsection 3.1 


we give, whenever possible, a characterisation of the different diagnosability problems. 


Finally in Subsection 3.3 we provide impossibility results that justify why characteri¬ 


sations were not given for some of the diagnosability notions. 


3.1 The logic pathL 

We define the logic pathL in this section. Similarly to Probabilistic Linear Time Logic 
(PLTL) |CY95j . pathL first defines a specification, then specifies a probabilistic condition 
over this specification. The main difference with PLTL is that pathL is based on the 
notion of path formulae instead of using atomic propositions. A path formula p is a 
predicate over finite prefixes of runs. Before defining their syntax formally, let us first 
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give some examples of path formulae. 

Example 3.6. Given a finite run p = q^a^qi ■ ■ .qk, let f be defined by f (p) = true if 
ai = f for some i < k. This path formula characterises the faulty finite rims. 

Let U be defined by ll(p) = true if there exists a correct signalling run p' with V{p) = 
V(p'). If this path formula is false, the current run is surely faulty. 

Let us introduce a more intricate path formula. For a G X*, we define firstf(rj) as 
the smallest value such that there exists a faulty run with observation a such that its 
prefix of length firstf(cr) is faulty, i.e. firstf(cr) = minjfc | 3p signalling run V(p) = 
a A pik is faulty} with the convention that min(0) = oo. Then the path formula 2U 
is defined by: 2U(e) = false and W(qoao ... q n +i) = true if fnstf (V(q^a® ... q n+ i)) = 
firstf('P(goOo • • • qn )) < oo. Every time this path formula is false, we increased the size 
of the greatest prefix that we are sure is correct. 

As shown by the path formula it for example, path formulae are extremely strong as 
they may depend on the global structure of the system, here by depending on the other 
existing runs with the same observation. This is far stronger than atomic propositions 
used in PLTL for example that only depend on the current state of the run. 

Formally, a path formula is either generated by a context sensitive grammar or 
equivalently its acceptance is decided by a linear bounded automaton |Kur64| . In other 
words, one has to be able to determine the truth of a path formula in linear space. 

Example 3.7. Among the examples of path formulae given in Example \3. 6} the most 
difficult one is 2U. Let us show how one can compute the truth of this path formula 
in linear space. We first define for a G X* and q € Q, the restriction of firstf to q: 
m(a, q) = min{A: | 3p 6 SR, last(p) = q A V(p) = cr A p^k is faulty} with the convention 
that nrin( 0 ) = oo. 

Let p be a finite run, a its observed sequence. If a = e, 2U(p) = false. Else 
a = o\...o n For every q G Q, we compute the values m(a,q) and m(a\ ... <r n _i, q) 
with the following algorithm: 


Algorithm 1 Computing the values of m 
1 : Input: pLTS A, a E £* 

2 : Output: {m{a< k ,q))k<\a\, q& Q 
3: for q G Q do 
4: m(e, q) <— oo 

5: for i = 1 to n do 
6: for q,q' £ Q do 

7: m{a\. . ,<Ji,q) <— oo 

8: if q => a f q' then 

9: m{a\ min(m(<7i... cq_ i,q'),i) 

10: if q q' then 

11: m(<Ji ...Vi, q') —i min(m(<7i.. . (Ti-±, q'),m(ai... ai-i,q)) 
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This algorithm uses linear space in n. Moreover, we have an equivalence between 
QIJ (p) = true and m&x qe Q{m(ai ... a n -i, <?)} < maXq e Q{m(<j, q)} < oo. Thus the truth 
of 2U can be decided by a linear bounded automaton. It is therefore a path formula. 

We now define the syntax of pathL. 

Definition 3.7. The syntax of a pathL formula is: 

<t> "= p | ~«j> | </>i A 4>2 | Off 

where p is a path formula. In the sequel we use the standard shortcut = —>0 —«f. 

A formula is evaluated at some position k of a run p = q^a^qi .... The prefix /?<& of 
p is defined by p<k = qo a oQi ■ ■ ■ c lk■ The semantics of pathL is inductively defined by: 

• p, k |= a if a(p<k), note that only the past of p is used; 

• p,k\= -xj) if p, k 4>] 

• p, k |= (fi A (f )2 if p , k |= <f\ and p, k \= 

• p, k j= O cf if there exists k' > k such that p , k' |= <f>. 

Finally p \= cf if p, 0 |= <f>. Due to the presence of path formulae (with no restriction) this 
language subsumes LTL and more generally any a;-regular specification language, i.e. 
any language that can be recognized by an cu-automaton such as a Rabin automaton. 

Proposition 3.6. The language generated by pathL subsumes uj-regidar languages. 

Proof. The language of a deterministic Rabin automaton is determined by a finite family 
of pair of sets (E tJ iy). It consists of the set of runs p for which there exists j 6 N such 
that p visits finitely often the states of Ei and infinitely often the states of F). We define 
the path formulae and such that 2b(p) = true iff last(p) £ E, and $i(p) = true 
iff last(p) £ Fi. The runs accepted by the Rabin automaton equivalently satisfy the 
formula \/^(O□ (—'^j) A (DO^i)). The language accepted by the Rabin automaton is 
thus generated by pathL. □ 

In order to reason about the probabilistic behaviour of a pLTS, we introduce the 
notion of qualitative probabilistic formulae: 

Definition 3.8. The syntax of a qualitative probabilistic formula of pathL is: P Mp ((/>) 
with cxi £ {<, >, =}, p £ {0,1} and f £ pathL. 

The semantics is obvious: A |= P Ixlp ((/>) if and only if¥({p £ D | p |= cf}) cx p. 

The set of Borel sets defined by pathL is closed by complementation since the comple¬ 
ment of the set of runs generated by a formula <f> is generated by the formula —xf. There¬ 
fore given a pLTS A and a formula cf , A |= P =1 (^>) iff A |= P =0 (-■</>) and A |= P <1 (0) iff 
A |= P >0 (-i<(>). The qualitative probabilistic formulae can thus be restricted to P =0 (c/>) 
and P >0 (<^>). 
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3.2 Logical characterisation of diagnosability 

We now exhibit qualitative probabilistic formulae if such that a pLTS A is diagnosable 
iff A |= fi. 

FF-diagnosability. We start with FF-diagnosability. This notion seems to be the 
easiest one as it focuses on faulty runs and its definition already uses a family of Borel 
sets. Indeed, we can prove a simple characterisation of FF-diagnosability using the path 
formulae f and 11 that we defined in Example |3.6[ 

Proposition 3.7. Let A be a pLTS. Then A is FF -diagnosable iff A |= P °(OD(f Alt)). 

Once a run becomes surely faulty, it cannot become ambiguous. Thus, informally, 
this formula means that a pLTS is FF-diagnosable if the measure of runs that are 
infinitely often faulty and ambiguous is equal to 0. These runs are the faulty runs for 
which the fault can never be claimed by the FF-diagnoser. It is thus natural to require 
their probability to be equal to 0 . 

Proof. We write E = {p E | OD(f Ail)} for the set of runs we are interested in. We 
further define, for every p E minF, E p = {p' &Ll\pPp'Ap'\= □!!} and for every 
n E N, E p = {p 1 E 17 | p < p’ A p’ |= □ n H} where p |= □ n </> if for every k < n, p, k |= fi. 
As p E minF and p',n |= f A 11 implies p'[0,n\ is faulty and ambiguous, we have for all 
n > 0 Ep C FAmb n+ |p|. Observe that E = l±J pem j n Fand that E p = n ne pj£' p . Thus 
W(E) = EpeminF nEp) and linw, P(££) = F(E p ). 

• Assume first that F(E) > 0. Then, there exists p E minF such that P (E p ) > 0. By 
definition, for every n > \p\ 0 P(FAmb n ) > P (E p ). Thus, A is not FF-diagnosable. 

• Assume now that P(i?) = 0. So, for every p E minF, P (E p ) = 0. Let us pick some 

£ > 0. Since F = UneN there exists no such that for every n > no, P(F\F n ) < |. Let 
R = {p E minF | \p\ 0 < no}. Pick a finite subset R' of R such that Y^ p &r\r' ^(/°) — §• 
Define K = |A? 7 1. Let ni be such that for every n > ni and every p E R', P (E p ) < 
Observe now that for every n > no, FAmb n - (F \ F n ) U 1+J p( zR\R> C(p ) U U P eR' Ep- 
Thus, for every n > m, P(FAmb n ) < | + | + = e. Since e is arbitrary, A is 

FF-diagnosable. ‘Q 

IF-diagnosability. I F-diagnosability focuses on infinite runs and FAmboo is not per 
se a Borel set. Obtaining a characterisation is thus more difficult. Thanks to Theo¬ 
rem |3.1| however, in finitely-branching pLTS the above characterisation also holds for 
I F-diagnosability. 

Corollary 3.2. Let A be a finitely-branching pLTS. Then A is IF -diagnosable iff A \= 
P=°(OD(f Ail)). 


lA-diagnosability. The assumption of finitely-branching pLTS is also needed in order 
to characterise lA-diagnosability. In addition to the path formula 11 we also use the path 
formula 2U defined in Example |3.6| page 78 
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Proposition 3.8. Let A be a finitely-branching pLTS. Then A is IA -diagnosable iff 
A \= P =0 (On(it A 2U)). 

Intuitively, as a faulty run cannot become correct, if a run does not satisfy it once, 
then p |= <>□ (-ill). Thus, p \f= On (it A 2U) if either (1) it does not satisfy it at least 
once, meaning p is surely faulty, or (2) does not satisfy 2T infinitely often. In the latter 
case, the intuition is the following. Infinitely often firstf has increased. If there exists 
a faulty run pj with V(p) = pp, then firstf is bounded, on the prefixes of V{p), by the 
length at which pf becomes faulty. Thus no faulty run exists with observation V{p) 
and p is surely correct. 

Proof. In order to prove formally the adequacy of the formula, it is enough to show that 
p E II is ambiguous if and only if p \= Od(it A 2T). We focus below on correct runs; the 
case of faulty runs can be treated in a similar, and even simpler, way. 

• Let p E CAmboo. Since p is ambiguous, there exists a faulty run p' such that V(p') = 
V(p). Let ko be such that p^ kQ is faulty. Thus for all k > ko, firstf (V(pik)) ^0 and 
in addition it is non-decreasing. So there exists some k\ > ko such that for all k > k\, 
firstf (V(pik)) is constant. We thus obtain p \= OD2U. Moreover, since p |= □ if, we 
conclude that p |= OD(ll A 2U). 

• Conversely, let p be a correct run such that p (= OD(ilA2H). Thus there is a position 

ko such that for all k > ko, p, k \= 2U. In particular, by definition of 2U, for all k > ko, 
there is a hnite signalling run such that V(p'^) = V(p^k) an d is faulty. 
Consider the tree of these runs p'^ by merging the common prefixes. This tree is 
finitely branching and infinite. By Konig’s lemma, it must admit an infinite branch, 
corresponding to a run p' with V(p') = V(p) and p'^ kQ faulty. We deduce that p is 
ambiguous. □ 


1-b 



Figure 3.16: An infinitely-branching lA-diagnosable pLTS which does not satisfy 
P =0 (On(ilA2H)). 


The pLTS of Figure [3.16 illustrates the need for the finitely-branching assumption 
The set of unobservable events is {u, f}. Observation b occurs in 


in Proposition 3.8 


every infinite correct run, while the observed sequence of the single infinite faulty run 
is aP. This pLTS is thus lA-diagnosable. However, it does not satisfy P =0 (OD(il A 2U)) 
since the unique infinite faulty run has probability b and satisfies at the same time 
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□211, by unicity, and □!(.. Indeed for every n £ N, there is a correct signalling run with 
observed sequence a n . 

Observe that the sets of runs specified by the characterisations of FF-diagnosability 
(On(fAil)) and lA-diagnosability (OD(ilA22J)) are F a sets, i.e. countable unions of 
closed sets. 

3.3 Non-expressivity results 

We chose to use a logic where the structural and probabilistic aspects are separated. 
This brought some advantages, but also weakens the expressibility. As a consequence, 
approximate notions of diagnosability for which the probabilities are important to define 
the ambiguity cannot be characterised. We prove this formally by showing that there 
is no Borel set E and F such that neither A |= P = 0 (A) nor A |= P > 0 (A) characterises 
1/2 FF-diagnosability. 

Proposition 3.9. There exists a finitely-branching LTS A such that for every Borel 
sets E and F of runs, there exists a pLTS A = (A, P) such that: 

• either A is 1/2FF -diagnosable and P. 4 (A) > 0; 

• or A is not 1/2FF -diagnosable and P g(E) = 0. 
and 


• either A is 1/2FF -diagnosable and P .4 (A) = 0; 

• or A is not 1/2FF -diagnosable and P .4 (A) > 0. 

This proposition is proved by constructing a family of pLTS whose underlying LTS 
is the same, thus the Borel set we construct is the same for every member of this family. 
However the probabilities can be appropriately chosen in order for the pLTS to model 
or not the formula in contradiction with its diagnosability. 


Proof. Consider the LTS A = {Q,qo,Yi,T) defined as follows and let the set of unob¬ 
servable events be T, u = 

• Q = {qo,qf,q c }, 

• £ = {a, b, f, u}, 


• T = {(g 0 , u, q c ), (g 0 , f, <?/), (<A, a, g c ), (g c , b , g c ), (q f , a, q f ), (q f , b , q f )}. 

We consider a family of pLTS, represented in Figure 3.17| with underlying LTS A. Given 
a pair of probabilities (pi,p2), we define the pLTS A( pitP2 ) = (A, P( pilP2 )) in which 
^*(pi ,P 2 ) (l7o, f, qf ) ^*(pi,P 2 ) q°) 1/2, f*(pi,p 2 )(?c, n, qc) Ph ^*(pi ,P 2 ) (?c, b, g c ) 

1-Pi. P (pi,p 2 )(^/> a >9/) =P 2 and P( Pl , P2 )(g/, b, q f ) = 1 - p 2 . 

First note that *4( Pl ,p 2 ) is l/2FF-diagnosable iff p\ p 2 - This can be established 
similarly to what was done in Example 3.5 page 68 In fact one can show that A( pitP2 ) 
is AFF-diagnosable iff p\ P 2 - 
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a,P 1 



b, 1 ~Pi 



Figure 3.17: A family of pLTS whose underlying LTS has no appropriate characterisa¬ 
tion for l/2FF-diagnosability. 


Let E be an arbitrary Borel set over the set of runs of A. If there exists a probability 
value p such that A ( p>p ) (= P =0 (£') we have the first part of the result. Else, let p and p' 
be two probabilities with p p' . As the probabilities of the runs of E in M( PjP /) do not de¬ 
pend simultaneously on p and on p\ P_ 4 (pp/) {E)+F A(pl p) (E) = P_ 4 (pjp) (E)+F A{p , y) (E). 
Moreover, for every probability p", -4( p ", p ") \f= P =0 (.E), thus either P _4 {E) > 0 or 

P_ 4 (p , ( E ) > 0 which concludes the first part of the proof. 

Let F be an arbitrary Borel set. If there exists a probability p such that M( PiP ) \= 
P >0 (A) we have the first part of the result. Else, let p and p' be two probabilities 
with p ± p'. As before, P.4 (Pil/) (-F) + P -V, p) ( F ) = P A p ,p) ( F ) + P -V , p0 ( F ) = °- Thus 
both A( P)P /) |= P =0 (F) and A( p / )P ) [= P =0 (E) which concludes the second part of the 
proof. □ 

For exact notions of diagnosability, intuitively, FA-diagnosability would be in be¬ 
tween FF-diagnosability and lA-diagnosability in terms of complexity. Surprisingly we 
showed that FA-diagnosability does not admit such a characterisation: there is no F a 
set E such that a pLTS A is FA-diagnosable if and only if A (= P =0 (E). 

Proposition 3.10. There exists a finitely-branching infinite LTS A such that for every 
Ffj set E of runs, there exists a pLTS A = (A, P) such that: 

• either A is FA-diagnosable and P_ 4 (E) > 0; 

• or A is not FA-diagnosable and P a(E) = 0. 

This proposition is proved in a similar fashion as Proposition |3.9[ it is only more 
involved. The family of pLTS we construct has infinitely many states and infinitely 
many parametric probabilities. These probabilities can be chosen in order to give a 
positive value to either the limit of the probability of CAmb.„ or to the given F a set. 

Proof. Consider the LTS A = (Q,qo,E,T) defined as follows: 

• Q = {/i; Qf} U {(R I i e N}; 

• £ = {a, b, c, u, f}; 

• T = {(q 0 ,u,q f ), (q 0 ,u,q i), (q f ,a,q f ), (q f ,b,q f ), /i), (/i, 6 ,/i), (/i,c,/i)} 

U {{q u a, q i+ i), (q u b, 9i+i)}*>i; 
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• = {f, 7t}. 



Figure 3.18: A family of pLTS whose underlying LTS has no appropriate characterisa¬ 
tion of FA-diagnosability. 


We consider a family of pLTS, represented in Figure |3.18 with underlying LTS A. 
For p = (p n )n> 1 a sequence of probabilities, we define the pLTS A p = (A, P p ) in 
which for every n > 1 the probability that W occurs from state q n is P p (g„, b, q n +i) = 
p n , the probability that ‘a’ occurs from state q n is P p (q n , a, q n +i) = 1 — p n and all 
other probabilities are independent of p: P p (qo,u, qf) = P p (qo,u,qi ) = P p (f\,b, fi) = 

P p (/i,c,/i) = P p (g/,a,g/) = P p (g/,fc,g/) = P p (g/,f,/i) = l- 

Observe that a faulty run almost surely produces a ‘c’, so that hin^^oo P(FAmb ra ) = 0. 
Moreover, as the ambiguous runs are the ones ending by a l b\ the probability to be 


ambiguous after n observations on the leftmost part is ^ and on the rightmost part 
is \p n P(CAmb n ) = p n + ^ 7 pr-. Therefore, A p is FA-diagnosable iff lim n _^ p n = 0. 


2 yn ^ v 11 — yn \ 3 

Let E be an arbitrary F, 
lim n — >ocp n = 0 and write P- 


set. We pick some FA-diagnosable A p i.e. 
for the probability measure it induces. If 


such that 

V(£) > 0 


we are done. Assume thus that ¥ P (E) = 0. In order to define a second pLTS, via p 7 , 
consider an infinite increasing sequence {rij}j< 1 and let for n £ [nj}j<i ■ p' n = Pn and for 
n E { n j}j>U p' n = \• Due to the sub-sequence p' n . = >A P ' is not FA-diagnosable. The 
sequence {nj}j< i depends on P p and will be dehned after some preliminary observations. 

Let F = {p | qouqi A p }. Denoting P p / the probability measure of the second pLTS, 
observe that P p /(E \ F) = P p (E \ F) = 0. 

EnF = U 

with \p\ — n, Om, 71+1 F Om,n and O m ri C Om+x^n- Denote F m — f^ n gj^O?n,n For all tti, 
lim, woo Pp(O m , n ) = Pp (E n Fm) < P p (E FI F) = 0. 

• m is chosen such that for all n > ni, p n < 5 . Observe now that for all rij, 


J p >(E \ F) = P p (E \ F) = 0. Using the above discussion, the F a set 
meN flngN where for all m, n, O m>n is a disjoint union of cylinders C(p) 

= flraSN Om,n 


, _ 1 _ 1 

Pn > 2 2 p n 


, ,1 1 

Pn, and 1 - Pn, = 2 - 1 - Vn i - " Pn >>' 


By definition of P p /, since O m>n is a disjoint union of cylinders C(p) with \p\ = n, 
applying inductively the previous inequalities, for all n such that nk < n < nk+i 
(denoting no = 0): 


Pp'(Om,n) < 


Pp(Om,n) 

Ui< j<k P n j 


(3.1) 
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• Assume that we have chosen n ±,..., n^. Since lim n ^. 00 P p (Ofcn) = 0, there exists 
rik+i > nk such that P P (Ofc,n*, +1 ) — FI 1 < Pn : , ■ We choose such an index. 

Equation (3.1) now implies that for all m < k, 


Thus for all m, 


Since E n F = |J 


mG N m ’ 


Pp' (Om,n fc+ i) A Pp' {Qk,n k+1 ) A ^k' 


P p >(F m ) = hm P p '(O m ,„ fc+1 ) = 0. 

fc—»oo 

y {E PI F) =0 and so P p /(.E) = 0. 


□ 


Beyond Proposition 3.10[ we conjecture that the impossibility also holds for arbitrary 
Borel sets. 

Proposition 3.10| only shows the non-existence for characterisations that requires a 
null probability of the given set. There could thus still exists a characterisation asking for 
a positive probability. In fact, such a characterisation does not exist. This impossibility 
is even stronger than the one of Proposition 3.10 as we show that a positive probability 
characterization cannot exist whatever the Borel set (and not only F a ). 


Proposition 3.11. There exists a finitely-branching LTS A such that for every Borel 
set E of runs, there exists a pLTS A = ((A, P), E 0 , V) such that: 

• either A is FA -diagnosable and P. 4 (E) = 0; 

• or A is not FA -diagnosable and P a{E) > 0. 


This proof is similar to the previous one, albeit with an even more complex family 
of pLTS. Indeed, here, instead of having a parametric probability pi for every i G N we 
have one for every word w 6 {a, b}*. 


Proof. Consider the LTS A = (Q, qo,Y,,T) defined as follows: 


• Q = {h,Qf, qo} u {q w | w e (a + 6 )*}; 

• £ = {a, b, c, u, f}; 


T = {{q Q ,u,q f ), (qo,u,qi), (q f ,a,q f ), ( qj,b,q f ), («?/,f,/i), (/i, 6 ,/i), (/1,c,/1)} 

U {(g W) a i Qwa)i (q dwb) } w£(a-\-b)* 5 

Y, u = {f,«}. 


We consider a family of pLTS, represented in Figure 3.19[ with underlying LTS A, 
parametrised by a mapping p : (a + b)* —> (0,1). Let A p = ((A, P p ), S G , V) be the 
pLTS such that the probability that l b J occurs from state q w is ~P{q w ,b,q w f) = p(w), 
the probability that ‘a’ occurs from state q w is P(q w ,b, q wa ) = 1 — p(rc) and all other 
probabilities are independent from p: P p (qo,u,qf) = P p (qo,u,qi) = P p (/i,6,/ 1 ) = 
P p (/i,c,/i) = P p (q f ,a,q f ) = P p (<?/,M/) = Pp(9/,f,/i) = In the sequel, for 
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Figure 3.19: Another family of pLTS whose underlying LTS has no appropriate charac¬ 
terisation of FA-diagnosability. 


convenience, we also write p (w,b) for p(io), and define p(io,a) = 1 — p (w), so that 
P (q w ,a,q wa ) = P (w,a). 

Word w can be decomposed into letters w = u;[l] ... w[n\, and we give notations for 
factors: w[l,k] = to[l] .. .w[k] with the convention that rc[l,0] = s. Finally we define 
P p (w) = rii<fc<nP(™M “ 1], it)[fc]) , as the probability to read w from q £ . Since 

lim^oo P(FAmb n ) = 0 and P(CAmb n _i) = S«j||u;|=n-i P(^> &) + we deduce that 
A p is FA-diagnosable iff lim n _>oo J2\ w \=n-i p( w i = °- 


Let E be an arbitrary measurable set. Pick some pLTS Ap which is FA-diagnosable, i.e. 
with lim n _>-00 ^| )j) | =n _ 1 p(ru, 6) = 0. If P P (E) = 0 where P p is the probability of this 
pLTS, we are done. Assume therefore that P P (E) > 0. Let F = {p \ qouq E C p} be the 
set of runs starting with a u-transition to q e . Denoting P p / the probability measure of 
any other pLTS A p ', observe that P p / (E \ F) = P p (E\ F). So, if P p (A \ F) > 0, then 
by picking any non FA-diagnosable (A, P p /), we are done. So assume P p (E \ F) = 0 
which implies P p (A n F) > 0. The probability being an inner regular measure (recall 
page [37]), there exists a closed set G C E n F with P p (G) > 0. 


Definition 


2.4 


If G = F then P p /(G) = P p (G) = \- In this case, we can therefore conclude by picking 
any non FA-diagnosable pLTS Ap'. 


Assuming G C F, since G is closed, there is some cylinder C(p) with p = qouq e ... q w 
such that G <1 C(p) = 0. Then we define the pLTS „4 p ' as the pLTS „4 P except that 
for every w A w' and every x G {a, b}, p \w',x) = Thus for every n > |tc|, 

Yl\ w '\=n p'( w 'i b) > Pp 2 ^^ . So Api is not FA-diagnosable. On the other hand, P p /(F?n 
F) > Pp'(G) = Pp(G') > 0. a 


With Proposition |3.10| and Proposition |3. 11 1 FA-diagnosability appears to be the most 
complicated of the exact diagnosability notions. Indeed, it cannot be characterised by 
any set of at least the first two level of the Borel hierarchy contrary to the other notions. 
This confirms the intuition that was raised when defining the diagnosers associated 
with each notion of diagnosability. For approximate notions of diagnosability, the non- 
expressibility result of Proposition |3.9| clearly shows that studying these notions requires 
a characterisation that intertwines structural and probabilistic elements. 
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4 Conclusion 

In Section [lj appropriate diagnosers were associated with notions of diagnosability. 
This gave another point of view on the notions of diagnosability and raised the question 
of the memory that is necessary for a diagnoser. We showed that for approximate 
notions of diagnosability, unbounded memory may be necessary even for finite systems. 
However for exact notions of diagnosability, every example we gave used finite memory. 
In Chapter |4j we study diagnosability of finite pLTS and establish what memory is 
needed for each notion of exact diagnosability. 

In Section [2] the links between the various diagnosability notions were established. 
This showed multiple interesting facts. In non-probabilistic systems, diagnosability 
only focuses on faulty runs as, for finitely-branching systems, if there is an infinite 
faulty ambiguous run, there also exists an infinite correct ambiguous run. However, 
when probabilistic models are considered, this symmetry is broken. Caring about both 
faulty and correct runs make for an entirely new notion. Moreover, probabilities do not 
affect correct and faulty runs in the same way concerning ambiguity: while IF- and FF- 
diagnosability are equivalent for finitely-branching systems, I A- and FA-diagnosability 
are not equivalent even for finite systems. Another interesting point, the uniformity 
requirement is not as important for exact notions of diagnosis than for approximate 
ones. Last, while AFF-diagnosability could appear to be close to FF-diagnosability as it 
forces an arbitrary high accuracy, the two notions are still very different. 

In Section [3j characterisations composed of a probabilistic requirement on a set of 
paths defined by a pathL formula were established, when possible, for the notions of diag¬ 
nosability. The notions which could not be characterised where the approximate notions 
of diagnosability (this is due to the sort of characterisation we were looking for) and FA- 
diagnosability. The latter is quite surprising as the definition of FA-diagnosability seems 
to be in between the ones of FF- and lA-diagnosability. Using model-checking techniques, 
one could derive an algorithm to decide a notion characterised by a logical formula. The 
absence of characterisation for approximate diagnosability and FA-diagnosability does 
not however mean that there is no algorithm for these problems. Indeed, we show in 
Chapter [4] that for finite pLTS, one can give a more specific characterisation of the 
exact diagnosability notions, including the FA-diagnosability. These characterisations 
have the same descriptive complexity and are used to obtain an algorithm. In Chap¬ 
ter [5j we study diagnosability for infinite-state systems and base our approach on the 
characterisations proved in the current section. 
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Chapter 4 

Algorithmic analysis of the 
diagnosability of finite pLTS 


In Chapter [3j we studied the notions of diagnosability in a general setting. More 
precisely, we showed that a diagnoser could be associated with each notion of diagnos¬ 
ability, we established the links between the different diagnosability definitions and gave 
a logical characterisation of multiple versions of exact diagnosability. With additional 
assumptions, stronger results can be established: characterisations can be refined and 
efficient algorithms can be designed. One of the usual restriction is the finiteness of 
systems. Many real systems can be represented with finitely many states (a vending 
machine, a Pac-Man game, ...). In fact, most of the systems which interaction with 
the environment only requires to keep in memory a finite number of events would fit 
in such a framework. For example, let us consider a server that receives and processes 
requests. If the server uses a finite memory, then after memorising a fixed number of 
yet unprocessed requests, the next request might be discarded without being processed 
due to a stack overflow. Such servers can be represented with finitely many states. 
Servers able to memorise an unbounded number of requests is dealt with in Chapter [5j 
When considering finite systems, we immediately get some results as consequences of 
the ones of Chapter [3] For example, any finite system being finitely branching, accord¬ 


ing to Theorem |3.1| page 73, IF-diagnosability and FF-diagnosability, two diagnosability 
notions focused only on faulty runs, are equivalent. As another example, every logical 
characterisation given in Section [3] of Chapter [3] applies. 

The first step is to establish what additional results can be obtained for diagnos¬ 
ability when we only study systems with finitely many states. These systems can easily 
be represented and have important properties that do not hold in the general case. For 
example, there is a finite number of probability values in the system since there is a 
finite number of transitions. This is not true in the general case as can be seen in the 
example of Figure 3.11 page 72 Using these additional properties, we establish new 


characterisations of the diagnosability notions in Section [l] These characterisations are 
not given as probabilistic logical formulae but as conditions on the structure or on the 
probabilities of the language observed in the system. More precisely, the characterisa- 
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tions of the exact diagnosability notions are purely structural ( i.e the probability values 
do not matter): they rely on a variant of the determinisation of the system. For the 
approximate diagnosability notions, the probabilities matter, and cannot be removed 
from the characterisation. The given characterisation relies on the comparisons of the 
probability of sets of observed sequences that can be observed from pairs of states. 

Once the characterisations are established, we apply them to get as many algorithmic 
and complexity results as possible. One such application, is to provide an algorithm 
to decide the diagnosability of a system. Indeed, given a “simple” characterisation, 
one can easily check if it holds on a given system. Checking a characterisation can 
be done with many different algorithms, some being more efficient than others. The 
efficiency of an algorithm is described by the complexity class the problem belongs 
to, in computational complexity theory. When possible we provide algorithms that are 
optimal with respect to the standard computational classes. For the other diagnosability 
problems, we establish that the associated decision problem is undecidable. This is done 
in Section [2] 

When a system is diagnosable, there exists a diagnoser for this system. The possible 
diagnosers use different amount of memory, give their verdict more or less quickly, etc. 
In Section [3j we show how to automatically construct a diagnoser for systems that 
are exactly diagnosable. The diagnosers we build have optimal memory and give their 
verdict as soon as possible. The constructions of the exact diagnosers are based on 
the characterisations given in Section [l] This strengthens the importance of these 
characterisations. 

This chapter develops and extends some of the results from |BHL141 IBHLlba] . 


1 Characterisations of diagnosability 


In this section, we establish characterisations for the different notions of exact diag¬ 
nosability and for one notion of approximate diagnosability. These characterisations 
strongly rely on the restriction to finite state systems. Therefore they are easier to 
express and check, but in general cannot be adapted to more general cases as will be 
seen in Chapter [5] As a direct consequence of the finite-state restriction and of Theo¬ 
rem |3Tj page |73| FF-diagnosability and IF-diagnosability coincide. So we only consider 
FF-diagnosability in the rest of this chapter. 

For all the exact diagnosability notions, the methodology is similar. We first con¬ 
struct an ad hoc deterministic automaton which gathers all the information needed for 
the diagnosis, by tracking possible correct and faulty executions. Secondly, we build 
the product of the original pLTS with this deterministic automator0 Diagnosability 
can then be characterised on the product by graph-based properties. 

For approximate diagnosability, we show that the diagnosability notions can be 
characterised relying on the distance 1 problem for labelled Markov chains (LMC). This 
problem, recalled in Chapter [2j receives as input two LMC and asks for the existence 


1 Using such a product to enrich the initial model was mentioned as an usual technique page 
There, the deterministic automaton was called belief automaton. 
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of an event, that is almost sure in one LMC, and has null measure in the other. It was 
shown to be decidable in PTIME [ICK14| . 

1.1 Exact diagnosis 

The deterministic automata we build are variants of the deterministic Biichi automaton 
introduced in [HHMS13] . that accepts the infinite unambiguous observed sequences. 
The latter tracks the subsets of possible states reached by signalling runs associated 
with an observed sequence. It looks like the on-the-fly determinisation of A viewing 
unobserved events as silent transitions. However, in view of the forthcoming character¬ 
isations, the subsets of correct and faulty states are divided in three sets: U, V and W. 
The intuitive meaning of these sets is the following one: 

• A state q belongs to U, if there is a correct signalling run with the current observed 
sequence ending in q; 

• A state q belongs to V U W if there is a faulty signalling run with the current 
observed sequence ending in q. 

• The partition between V and W ensures that for all q G V, q' G W and p a faulty 
run ending in q, there exists a faulty run p' ending in q' with an earlier fault than 
the fault of p. In other words, V and W contain the states reached by faulty runs, 
while W keeps track of the runs that have been faulty for the longest time. 

W corresponds to the set of faulty states for which the ambiguity with the correct states 
of U has to be resolved (when both are not empty), while V corresponds to a waiting 
room of states reached by faulty runs that will be examined when the current ambiguity 
is resolved. 

Before giving the definition of the deterministic automaton associated with a pLTS 
and for sake of readability, we define for two sets of states U and V and an observation 
a£S„ the set of states 

updatefaulty(t/, V, a) = {q \ 3q G U, q' q} 

U {q | 3q' G V. <{ => a q}. 

This set contains the states reached from U by a faulty signalling run of observation a 
and the ones reached from V by a signalling run of observation a. 

Definition 4.1. Given a pLTS A, the deterministic automaton associated with A is 
Obs(A.) = {S, so, A, F} where 

• so = ({<Zo},0,0) the initial state ofObs(A); 

• the states and transitions of the deterministic Biichi automaton 0bs(„4) are in¬ 
ductively defined by: 

Given (U,V,W) a state ofObs(A) and a G £ 0 , there exists a state (U',V r , W') G S 
and a transition (U, V, W) —> (U', V' , W') in A as soon as: 
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1. {qeUUVUW \ g^ a } ^0, 

2 . U' = {q\3q' eU,q' ^ a c q}, 

3. If W = 0 then V’ = 0 and W’ = updatefaulty([/, V, a), 

4 ■ IfW / 0 then W’ = updatefaulty(0, W, a) and V' = updatefaulty([7, V, a)\ W’; 
• the set F of accepting states consists of all triples (U, V, W) with U = 0 or W = 0. 


When U = 0 , the current signalling run is surely faulty, since U tracks the possible 
states after a correct run. When W = 0 the current signalling run may be ambiguous (if 
V A 0 ) but the “oldest” possible faulty runs under scrutiny have been discarded. Hence, 
any infinite observed sequence of A passing infinitely often through F is not ambiguous 
(either it is surely faulty, or ambiguities are resolved one after another). 


Example 4.1. Let A be the pLTS of Figure\4-1\ We represent the associated determin¬ 


istic automaton 0bs(„4) in Figure 4-%< where accepting states for the Biichi condition 
are doubly framed. 



a abb 

Figure 4.1: An IA and FF-diagnosable pLTS which is not FA-diagnosable. 


Let us consider the observed sequence a u which has probability 1/2 in the pLTS. The 
initial state of Obs(^l) is ({go}, 0,0} meaning that the initial state of the pLTS is go 
and no fault occurred so far. Then, observing ‘a’ we reach ({f/i, g2}, 0, {/2D meaning 
that from go, we can reach q\ and q -2 with correct signalling run of observation ‘a’ and 
/2 with a faulty signalling run of observation ‘a ’. As this faulty run is the oldest one, 
it is added to W. The second observed ‘a’ leads to the state ({gi, q 2 }, {/ 2 }, 0)- The 
run that ended in fo previously cannot be extended by observing an ‘a’, thus the set W 
which tracked this run (as the only oldest faulty rim) is empty. However a new fault can 
be created ending in state fi, this fault being made while W was tracking other faults, 
it is not considered “old” and thus is added to V. This state is therefore an accepting 
state. Observing yet another ‘a’ leads back to ({gi, g 2 }. 0, { ^ 2 } )- The run tracked by V 
disappeared, a new fault was created and as W was empty it is considered a “old” fault 
and f -2 is put in W. As the path of Obs(»4) corresponding to the observed sequence oF 
alternates between these two states infinitely often, it visits infinitely often a state where 
W = 0 , thus a u is accepted. 

For the other observed sequences, a ‘b ’ can only be observed in a faulty state, therefore 
any observed sequence containing a ‘b’ is surely faulty. This can be seen in Obs(^l) as 
any path containing a ‘b’ ends in one of the five rightmost states, all of which have 
an empty set U (the first component corresponding to the correct reachable states). As 
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b 



Figure 4.2: The deterministic automaton associated with the pLTS of Figure 4.1 
states that are framed twice are accepting for the Biichi condition. 


The 


any state with U = 0 is accepting, every infinite observed sequence containing a ‘b’ is 
accepted. 


The next proposition recalls the main property of this automaton. 


Proposition 4.1 ( |HHMS13j ). Let A be a finite pLTS. Then the deterministic Biichi 
automaton 0bs(„4) accepts exactly the infinite unambiguous observed sequences of A. 


Example 4.2. As seen in Example \ f.l\ every observed sequence of the pLTS of Fig¬ 
ure f.l is accepted by the associated deterministic automaton. According to Proposi¬ 
tion 4.1. this means that there is no infinite ambiguous sequence in this pLTS, thus it 


is FF- and FA -diagnosable. 


1.1.1 FF-diagnosability 

As explained earlier, for each diagnosability notion, we consider a variant of Obs(A). 
For FF-diagnosability, we only need to remove the ambiguity for faulty runs So we can 
omit the faulty sets of states V and W. We write FF(A) for the resulting simplified 
automaton, called FF-automaton, obtained from Obs(A) by only considering the 17- 
component of states. 

Example 4.3. Figure \fi^ illustrates this construction on the pLTS of Figure \4-1\ This 
automaton reflects that once b happens, the current signalling run is surely faulty. Thus 
the set of possible correct states is empty (state S 2 ). 

To recover the stochastic behaviour of A which is not reflected in FF(A), we now 
define the pLTS Aff = A x FF(7l) as the product of A and FF(A) synchronised over 
observed events. 

Definition 4.2. Given a pLTS A = {Q, qo,T,,T) associated with the FF-automaton 
FF(A) = {S, sq, A, F}, we define Aff = {Q', {qo, {<?o})) T') where: 
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b 



a, b 


Figure 4.3: The FF-automaton of pLTS of Figure 4.1 


• Q' = Q x S; 

• ((<?, B),a, (</, B )) € V iff (q, a, q') € T and 

— either a G and B = B', 

— or a £ X 0 and f/iere exists a transition B -0 B' in A. 


Since FF(„4) is deterministic and complete, Mff is still a pLTS, with the same 
stochastic behaviour as A. More precisely, there is a bijection between the runs of A 
and the runs of FF(*4). A run and its image by the bijection have the same observation 
and the same probability. In addition, the [/-component of a state (q, U ) of Mff stores 
the relevant information w.r.t FF-diagnosability of the observed sequence so far. 


Example 4.4. Carrying on with the example pLTS of Figure \4■ 1\ Figure 4-4 shows the 
resulting product pLTS. Observe that it has two bottom strongly connected components 
(BSCC), consisting of the absorbing states (qi,s\) and (/ 2 ,S 2 )- 


(qusa^-- 



“00 


u 00 f 
-- 02 , 0 ---- 


a b\ i s 2 ) a J^^0^— 

°C0- f y ^ 6 



Figure 4.4: The synchronised product of the pLTS of Figure 4.1 and its FF-automaton. 


In a finite pLTS almost all runs ends in a BSCC |BK08| . and FF-diagnosability 
is a property of runs expressed as an almost sure event. So, the characterisation of 
FF-diagnosability can be stated on the BSCC of Mff- 

Proposition 4.2. Let A be a finite pLTS. Then A is FF-diagnosable if and only if Aff 
has no BSCC containing a state ( q , U ) with q E Qf and U 0 . 
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The proof relies on two elements. First, a run will almost surely reach a BSCC of 
the pLTS. Second, for a faulty run to be ambiguous, the set U contained in the state 
in which the run ends must not be empty. Therefore if every BSCC corresponds to 
either correct runs (he. q G Q c ) or faulty unambiguous runs (he. U = 0), the pLTS is 
FF-diagnosable. 

Proof. Suppose first that there exists a reachable BSCC C of Aff and a state s = (q, U ) 
in C such that q G Qf and U A 0- Let p be a signalling run leading from the initial 
state So °f Aff to s. Now, for every state s' = (q',U') G C, necessarily q’ G Qf and 
U 1 yh 0, because C is strongly connectec0 So for every signalling run p' that extends 
p, writing s' = (q r , U') for the state p' leads to, there exists a correct signalling run p" 

p" 

such that V{p") = V(p') and qo d—F q" with q" G U'. As a consequence the observed 
sequence V{p") is ambiguous in Mffi and f° r every n > \p\ 0 , P(FAmb n ) > P(p). As A 
and Aff have the same ambiguous observed sequences and the associated runs have the 
same probabilities, A is not FF-diagnosable. 

Suppose now that for every state s = (q, U) of a BSCC C, either q G Q c , or U = 0. 
This property is in fact uniform by BSCC: for every BSCC C, either for every state 
(q, U) G C, q G Q c , or, for every state (q, U) G C, U = 0. This is a straightforward 
consequence of C being strongly connected. Moreover, if a run p reaches a pair (q, U ) 
then q G Q c implies U A Indeed, let p' be the greatest signalling run prefix of p. p' 
ends in a pair (q', U') where U' = U as V(p) = V(p'). Moreover if q G Qc then q' G Q c , 
therefore q' G JJ implying that U A Therefore in Aff th e BSCC are partitioned in 
correct ones, in case all g-components of states in C are correct, and faulty ones, in case 
all 17-components of states in C are empty ensuring unambiguity of faulty runs ending 
in a BSCC. Since runs almost surely leave the transient states and reach a BSCC, this 
implies that lim n ^. 0O P(FAmb n ) = 0. D 


As a consequence of this characterisation, we establish the equivalence between FF- 


and uniform FF-diagnosability for finite pLTS, claimed in Theorem 3.1 


Proposition 4.3. Let A be a finite pLTS. If A is FF-diagnosable, then it is uniformly 
FF-diagnosable. 


In a finite FF-diagnosable pLTS, thanks to the characterisation given in Proposi¬ 
tion [L2] we know that faults can be detected at worst when the run reaches a BSCC. 
The proof consists in showing and using that the speed at which a BSCC is reached is 
uniform from any state. 


Proof. Let A be an FF-diagnosable pLTS. Given a run p of A, let ppF be the corre¬ 
sponding run in Aff'- the states in pff extend the states appearing along p by subsets 
of possible correct states after the corresponding prefix of the observed sequence V(p). 
Let S bscc denotes the set of states of Aff that belong to a BSCC. Last, for every state 
(q, U ) of Aff and every n G N, denote by SRfi u the set of signalling runs in Aff of 
length n starting at (q, U ). 

2 Recall that the set Qf is absorbing 
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Let a > 0. Our objective is to get n a such that for every n> n a and every minimal 
faulty run p E minF: 

P. a({p' G SR n+ | p | o \ p<p' A CorP (V(p)) > 0}) < a • P (p) . 

We first exploit the almost sure convergence towards BSCC in Aff- For every state 
(q, U ) of Aff, the measure of runs starting in (q. U ) and avoiding all BSCC during n 
steps tends to 0, when n goes to infinity. Thus, given cr, for every reachable ( q,U ), 
there exists n q> u E N such that for every n > n q jj, 

P^ff({Pff e SR| last(pp F ) 0 S'bscc}) < °- 

We define n a as the maximum of n q) u over all states (q, U). 

Now let p be a minimal faulty run of A. and define ( q,U ) = last(/9pp). Since 
n a > n q ,u, Pyt F p ({Pff ^ SR^ | last(pp F ) 0 S'bscc}) £ Therefore, as A and „4 ff have 
the same probabilistic behaviour, 

p a({p g SR„ a+ | p | o | P < P A last(pF F ) 0 S’bscc}) < a ■ ¥(p). 

Thanks to the characterisation of Proposition |4.2| all states in BSCC reachable from 
(q,U) in Aff necessarily are of the form ((/,0). Therefore, if a finite run p FF reaches 
such a BSCC, p FF admits uo correct run with same observed sequence, and hence 
CorP('P(p F p)) = 0. Equivalently, CorP('P(p / )) > 0 implies last(p FF ) ^ S'bscc- Thus 

p a({p' € SR ncv+ | p | 0 \p<p' A CorP {V{p')) > 0}) < a ■ P(p) 

which shows that A is uniformly FF-diagnosable. □ 

1.1.2 FA-diagnosability 



Figure 4.5: The FA-automaton of the pLTS of Figure [4~Tj 


For FA-diagnosability, we again start from Obs(^4). Here, we need information about 
the ambiguity of both faulty and correct runs. Yet, we still do not need to keep all the 
information given by Obs(^l). Indeed, we can gather the V and W components into 
a unique set, that we again call V. In other words we keep the information on which 
faulty states could be reached, but not the distinction between “old” and “new” faulty 
runs. The resulting simplified automaton is denoted by FA(yl). 
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Example 4.5. Figure 4-5 illustrates this construction on the pLTS of Figure 4-1 As 
expected, the FA-automaton is a refinement of the FF-automaton: the U-component of 


a state in FA(A) corresponds to a state in FF(A). For instance, state S 2 of Figure 4-3 
is split here into S 2 , s 3 and S4. 

As we did for the FF case, we now define the pLTS Afa = A x FA(A) as the product 
of A and FA (A) synchronised over observed events (the definition has the same structure 


as the one of Definition 4.2 only using FA(A) instead of FF(A)). Afa is still a pLTS 
with same stochastic behaviour as A augmented with the relevant information of the 
observed sequence w.r.t FA-diagnosability. 


Example 4.6. Figure \ 4 fi 6 \ continues Example \4-5\ and shows the synchronised product 
for the pLTS of Figure \4-l\ 







a 91 , si 



a 92,«iV- 



Figure 4.6: The synchronised product of pLTS of Figure 4.1 and its FA-automaton. 


Again, FA-diagnosability is characterised through the BSCC of Afa- 

Proposition 4.4. Let A be a finite pLTS. A is FA-diagnosable if and only if Afa has 
no BSCC that: 


• either contains a state (q, U, V ) with q € Qj and U 0; 

• or contains a state ( q,U,V ) with q £ Q c and V 0. 


Note that the characterisation of FA-diagnosability is symmetric for correct states 
and E-component (resp. faulty states and [/-component). This reflects the symmetry 
of the definition of FA-diagnosability. 

The main difference between this proof and the one of Proposition 4.2 is that the 
second item is not uniform inside a BSCC: there may exists a BSCC containing two 
states ( 91 , U\, Ei) and ( 92 , U 2 , E 2 ) with 91,92 £ Q c , Vi = 0 and E 2 A 0. As a consequence 
it is harder to show that if a BSCC verifies the second item, then the pLTS is not FA- 
diagnosable. Instead of having every extension of the run to be correct ambiguous, 
the extensions only are ambiguous when they visit some particular states. However, 
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the probability that a run ends in such a state converges towards the steady state 
distribution of this state which is positive as the state belongs to a BSCC, contradicting 
the FA-diagnosability. 


Proof. To prove the left-to-right implication, we proceed by contraposition. If one 


assumes the first item holds, the same argument as in the proof of Proposition 4.2 
applies. Precisely, suppose that there exists a reachable BSCC C of Afa and a state 
s = ( q , U, V ) in C such that q € Qj and U A 0. Let p be a signalling run leading from 
the initial state so of Afa to s. Now, for every state s' = (q', U', V') £ C, necessarily 
q' £ Qf and U f A because C is strongly connected. So for every signalling run p' 
that extends p, writing s' = (q',U',V') for the state p' leads to, there exists a correct 

p" 

signalling run p" such that V{p") = V{p') and qo q" with q" £ U'. As a consequence 
the observed sequence V{p") is ambiguous, and for every n > |p| D , P(FAmb n ) > P(p), 
so that A is not FA-diagnosable. 

Suppose now that there exists a reachable BSCC C of Afa and a state s = (q, U, V) in 
C such that q £ Q c and V A 0. Since the pair (U,V) is unchanged by unobservable 
transitions, w.l.o.g we assume that s is the successor of some state of C by an observable 
event and we denote C' the set of such states. Observe that a signalling run that reaches 
s is ambiguous. Denote 7 Tj(V) the probability that a random run of length i ends in 
a state s'. In a finite DTMC, for every state s' of a reachable BSCC the Cesaro-limit 
ftoc(s') = lirrpj^oc l/(n + 1) X^=o 7r *( s/ ) ex ists and is greater than 0. For s' £ C' denote 
by p s p s the probability of an observable transition from s' to s. Then 


0 < ^ ^ TTqo {s')p s >,s < liminf —- V' «j(s) 

z ' n—>oo n + 1 z — 4 


s'eC" 


1=0 


where aj(s) is the probability that a random signalling run of length i ends in s. a.i 
differs from 7 r* by only considering signalling runs. From time 0 to time n. a run can 
be a signalling run at most n + 1 times. Thus: 


1 


n + 




< 


1 


i =0 


n + 1 


P(CAmb ; : 


i=0 


which implies that 


1 - ^ 

0 < lim inf-y^P(CAmbj) < limsupP(CAmb n ) . 

n-kx> n + 1 z —f n—too 

j=0 

In this case also, we conclude that A is not FA-diagnosable. 

The proof of Proposition |4.2| has established that a signalling run reaching a BSCC 
C where for every state s = ( q , U,V), q is faulty and U = 0, is surely faulty. Similarly a 
signalling run that reaches a BSCC where for every state s = (q, U,V), q is correct and 
V = 0, is surely correct. Thus an ambiguous signalling run must only visit transient 
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states. Since runs almost surely leave the transient states and reach a BSCC, this 
implies that: 

lirrisupF( FAmb n ) +P(CAmb n ) = 0 , 

n—>oc 

and therefore, the pLTS is FA-diagnosable. 0 

Let us emphasise that although there does not exist a simple logical characterisation 
of FA-diagnosability, in finite pLTS, it enjoys a characterisation that is similar to the 
one of FF-diagnosability. 


1.1.3 lA-diagnosability 

lA-diagnosability is the notion of exact diagnosability for which we need to use Obs(0) 
with no simplification. However, to stick to the presentation for the other diagnosability 
notions, we write here IA(„4) for Obs(0). As before, to come up with a characterisation, 
one builds 0 ia = A X IA(„4), the product of A and IA(0) synchronised over observed 
events. 


Example 4.7. Figure 4 A shows the synchronised product corresponding to the pLTS 


depicted in Figure 4-1 Among the BSCC, all the faulty ones (i.e. the ones reached 


after a faulty event) have U = 0, while si), (qi, the single one that is reached 
by a correct run, has a state (q±, s^) with W = 0. 



Figure 4.7: The synchronised product of the pLTS of Figure 4.1 and its lA-automaton. 
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We now establish a characterisation of lA-diagnosability on A\/\. 

Proposition 4.5. Let A be a finite pLTS. A is IA -diagnosable if and only if A\/\ has 
no BSCC such that: 


• either, all its states (q, U, V, W) fulfil q G Qf and U 0 ; 

• or all its states ( q , U, V, W) fulfil q 6 Q c and W 0 . 


Proof. This proof relies on Proposition |4.1[ An infinite run is not ambiguous if its 
observation satisfy the Biichi condition of Obs(M), therefore Mia will be lA-diagnosable 
if the pLTS almost surely satisfies the Biichi condition. As a run almost surely reaches 
a BSCC, every BSCC must contain a state that allows the diagnosis. 

Assume first that Mia has a BSCC with (at least) some state (q, U, V, W) with 

q E Qf and U 0. Using Proposition 4.2| A is not FF-diagnosable and thus not 

lA-diagnosable either, due to Theorem 3.1 Assume now some BSCC C of Mia has ah 
its states ( q , U, V, W) with q £ Q c and W 0. In particular none of these states are 
accepting for the deterministic Biichi automaton IA(M). Let p be a finite signalling run 
that hits C. By Proposition |4.1[ any infinite run p' that extends p is ambiguous. From 
q £ Q c we deduce that P(CAmb oc ) > P(p) > 0. Therefore A is not lA-diagnosable. 

Assume now Mia has no BSCC such that either, all its states (q , U, V, W) fulfil 

q E Qf and U 0, or ah its states (q, U, V, W ) fulfil q E Q c and W 0. First observe 

that in case some BSCC of Mia contains some state (q, U, V, W) with q E Qf and U 0, 
then ah its states satisfy the same constraints. Moreover, if some state (q , U, V, W) of 
a BSCC has q 6 Q c , then all states of this BSCC have their first component in Q c . 
Therefore, the condition can be reformulated as follows: all BSCC C of Mia satisfy: 


• either all states (q, U, V, W) of C fulfil q e Qf and JJ = 0 ; 

• or all states (q, U, V, W) of C fulhl q G Q c and some state {q. U, V , W) of C fulhls 
W = 0. 


Whatever the case, all BSCC contain (at least) an accepting state for the Biichi condi¬ 
tion of IA(M). Since all runs almost surely end in a BSCC and visit each of its states 
infinitely often, using Proposition |4.1| almost all runs of Mia are unambiguous. This 
proves that A is lA-diagnosable. D 

Surprisingly, while in general FA-diagnosability could not be characterised by a log¬ 
ical formula contrary to lA-diagnosability, restricted to finite systems, the characterisa¬ 
tion of lA-diagnosability is the more involved one. 


1.2 Approximate diagnosis 

We now turn to the characterisation of approximate diagnosis and particularly of AFF- 
diagnosability. The reason why we only consider AFF-diagnosability here will become 
clear in Subsection 2.2.1 where we show that ah other approximate diagnosability no¬ 
tions are undecidable. Our characterisation of AFF-diagnosability relies on the notion of 
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distance between two Markov chains with labels on the transitions. A labelled Markov 
chain (LMC) is a pLTS where every event is observable: E = E 0 . In order to exploit 
results of |CK14] on LMC in our context of pLTS, we introduce the mapping Ai that 
computes in polynomial time the probabilistic closure of a pLTS w.r.t. unobservable 
events and produces an LMC. Informally, the probabilities of all paths of A from state 
q to state q' with same observed sequence a E E 0 are gathered to obtain the probability 
in A4(A) to move from q to q 1 with label a. The transformation is formally defined 
below. For sake of simplicity, we denote by A q , the pLTS A where the initial state has 
been substituted by q. 


Definition 4.3. Given a pLTS A = (Q,qo,T,,T,P) with S = S 0 l±J E Mj the labelled 
Markov chain Ad(A) = {Q, qo, E 0 , T', P') is defined by: 


T' = {(q, a, q') \ 3p = q ■ ■ ■ aq' E SRi(M 9 )} (and so a E E Q ). 

for every ( q,a,q ') E T', P'(q,a,q') = F({p E SRi(M 9 ) | p = q---aq'}). 


Example 4.8. The LMC associated with the pLTS of Figure \4-l\ is represented in 
Figure f.8. The transition from qo to q± which was unobservable has been replaced by a 
transition labelled by ‘a The new transition has probability 1 /2 which is the product of 
the probability of the replaced unobservable transition (of value 1/2) and of the transition 
labelled by ‘a’ that followed (of value 1 ). A transition from qo to f 2 appeared, it replaces 
the run go'U^f/io.// which has probability 1/8. 


a, 1/8 



Figure 4.8: The LMC obtained from the pLTS of Figure 4.1 


Let E be a prospecij^Jof E)/ ( i.e. a measurable subset of E^ for the standard measure), 
we denote by the probability that prospect E occurs in the LMC A4. Given two 

LMC Aii and Ai 2 , the (probabilistic) distance between A4i and Ai 2 generalises the 
concept of distance for distributions. Given a prospect E, \F Ml (E) — P- A/<2 (E)| expresses 
the absolute difference between the probabilities that E occurs in Ai\ and in A4\. The 
distance between Ai\ and AI 2 is defined as the supremum over the prospects: 


li The term used in the literature is event. We differ here as we already use event for the letters 
labelling the transitions as established in Definition 


2.5 


page 


38 
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Definition 4.4. Let Mi and M 2 be two LMC over the same alphabet £ 0 . Then 
d(Mi, M 2 ) the distance between Mi and M 2 is: 

d(Mi,M 2 ) = supUP^H^-P^ 2 ^)! | E prospect o/S“} . 

Example 4.9. Consider the LMC of Figure \4-8[ called M\, and the one of Figure \f.9\ 
called M 2 ■ The prospect E = {a u ) has probability 1/2 in M\ and 0 in M 2 ■ Thus 
d{ M\, M 2) > 1/2. Moreover, M 2 can be obtained from Mi by deleting the state qi (and 
the associated transitions) and merging qo with q 2 ■ As a consequence, for any prospect E 
such that aF 0 E, P M *(E) = 2.P Ml (E). We thus have P^da"}) -P^ a ({a w }) = 1/2, 
P^ 2 W\{o"}) -P^i(E“ \{a“}) = 1 / 2 , and for any other prospect E, the difference 
is smaller or equal to 1 / 2 . Therefore d(Mi, M 2 ) = 1 / 2 . 



Figure 4.9: An example of LMC. 


The distance 1 problem asks, given two LMC M 1 and M 2 , whether d(Mi, M 2 ) = 1. 
The next proposition summarises the results of Chen and Kiefer on LMC, that we use 
later. 

Proposition 4.6 ([ CK14] ). 

• Given two LMC Mi, M 2 , there exists a prospect E such that: 

d{Mi, M 2 ) = P Ml (E) - P M2 (E). 


The distance 1 problem for LMC is decidable in polynomial time. 


The first item of this proposition states that the supremum is reached (and thus is 
In fact, given two LMC Mi, M 2 , the authors show that one prospect 

P^^n) 


a maximum 


reaching the maximum is E = {w G | lim 


>!}■ 


n —>00 P M2 (w< n ) 

We now use the notion of distance 1 to characterise AFF-diagnosability. Let us first 
consider a subclass of pLTS called initial-fault pLTS. Informally, an initial-fault pLTS 
A consists of two disjoint pLTS A? and A c and an initial state qo with an outgoing 
unobservable correct transition leading to A c and a transition labelled by f leading to 
A f (see Figure UrO). Moreover no faulty transitions occur in A c . In other words, if a 


fault occurs during a run of an initial-fault pLTS, it does so on the very first transition. 


Definition 4.5 (Initial-fault pLTS). A pLTS A = (Q,qo,E,T,P) is an initial fault 
pLTS if there exist two disjoint pLTS A? = (Qf,qf,E,Tf,Pf) and A c = (Q c ,q c ,Ti\ 
{f},T c ,P c } such that: 
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f - 

\ ’ 2 





V) 


Figure 4.10: Schematic representation of an initial-fault pLTS. 


• Q = {q 0 } w Qf w Qci 

• T = Tf l±l T c tfcl {{qo,u, q c ), (q 0} f, q f )} with u G 

• for every t G Tt, we have P (t) = Prft) and for every t G T r , we have P(f) 
and P((g 0 , u, qj)) = P {(q 0 , f, q f )) = 1/2. 


We denote such a pLTS by A = (qo,A-f,A c ). 

Example 4.10. The pLTS of Figure\4-ll is a simple initial-fault pLTS. 


a 5 

U, i 4 



P cit), 


Figure 4.11: A uniformly AFF-diagnosable initial-fault pLTS. 


Under the initial-fault restriction, we can get a simple characterisation for AFF- 
diagnosability as established in the next lemma. The idea of this characterisation is 
then extended to every pLTS. 

Lemma 4.1. Let A = (qo, A?, A c ) be an initial-fault pLTS. Then A is AFF-diagnosable 
if and only if d(A4{A-^), A4(A C )) = 1. 

We write P, Pj and P c for the probability measures of pLTS A, A? and A c . By 
construction of M(A^) and J\A(A C ), for every observed sequence a, p-W-- 4 1 (cr) = P/(<r) 
and similarly p jV1 (4 r: )( CT ) — p c (<j). In words, the mapping A4 leaves unchanged the 
probability of occurrence of an observed sequence. 

AFF-diagnosability is a property of identification (decide whether the run is faulty) 
of the finite runs that start by a fault with high probability, using the probability P. 
The distance 1 between AifA-f) and Ai(A c ) is also a property of identification (decide 
which LMC the run that produces the infinite observed sequence belongs to) of infinite 
observed sequences, using the probabilities Pj and P c . As mentioned above, these three 
probability measures are probabilities of parts of the pLTS A and are thus related. The 
proof therefore mostly consists in understanding the links between the two identification 
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properties: how to translate the set of finite runs identified by AFF-diagnosability into a 
set of infinite sequences and reciprocally. The intuition to establish the relation is that 
the prefixes of an infinite faulty run reveals the fault with arbitrarily high accuracy if 
and only if the associated infinite observed sequence “reveals” that the run belongs to 


Proof. Let us prove the equivalence, starting with the left-to-right implication. 

• Assume A is AFF-diagnosable. Then, for every e > 0 and every minimal faulty run p: 

lirn P {{p 1 G SR n+ | p | | p < p' A CorP(TV)) > e}) = 0. (4.1) 

n — 1 ^ 1 


Pick some 0 < e < 1. Applying Equation (4.1) on the minimal faulty run pf = qofqf 
with | V(p f )\ = 0, there exists some n G N such that: 


P({p G SR n | Pf A p A CorP {V{p)) >£})<£ ■ 


Let © be the set of observed sequences of faulty runs with observable length n and 
correctness proportion not exceeding threshold e: 


© = {o G E” | 3p G SR n ,V(p) = a A pp A p A CorP(cr) < e} . 

We define E = Cyl(©) to be the prospect consisting of the infinite suffixes of these 
sequences. Let us show that P C (E) < e/(l — e) and P f(E) > 1 — 2e. We have: 


P f(E) = 1 — 2 P({p G SR n | pp A p A CorP{T(p)) > £}) > 1 — 2e 


where the factor 2 comes from the probability 1/2 in A to enter A? that P f does not 
take into account contrary to P. 


Moreover, for every observed sequence a G ©, CorP((j) < e. Using the definition of 

CorP: 


CorP(cr) 


P({p G C Cl SR n | V(p) = rr}) 
p({p G SR n I v(p) = a}) 


Pc(g) 

p c (<r) +P/(cr) 


< £■ 


Thus, P c (<r) < P/(u). Hence: 


P c ( J E) = ^P c ( £ T)<^ r ^ 7 P / (cT) 
o-ee tree 1 




£ 

1 — £ 


Therefore d(M(A c ), M(A^)) >Pf(E) — P c (E) > 1 —e(2+j^). Since e was arbitrary, 
taking the limit when e goes to 0, we obtain the desired result: d(M(A c ), Ai(A^ )) = 1. 
Note that we did not exhibit the prospect that reaches the maximum but only a prospect 
e-close to it. The proof could be modified to use this maximum prospect, but it makes 
the proof unnecessarily more complicated. 

• Conversely assume that d(A4(A^), M(A C )) = 1. Thanks to Proposition 
exists a prospect E C Yff such that P f(E) = 1 and P C (E/) = 0. 

For every n G N, let © n be the set of prefixes of length n of the observed sequences 
of E: & n = {a G | 3a' G E, a A a'}. For every e > 0, we also define ©/ as the 


4.6 


there 
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subset of © n consisting of sequences whose correctness proportion exceeds threshold e: 
& n = {a £ & n \ CorP(cj) > e}. 

From P| ngN Cyl(6 n ) = E , we derive that lim n _ > . 0O P c (6 n ) = P C (E) = 0. Thus 
lirn^oo P c (©^) = 0. 

On the other hand, for every n £ N, 

P C (6 £ J = £ P c (<t) > TZr/f^ = • 

crG&f, ct£ 6 ® 

Since e is hxed, we have P/(6^j) < ^j^P c (©^), thus limn^oo P c (©fJ = 0 implies that 
lim ?woo P / (©=) = 0. 

Let p be a minimal faulty run and a > 0. There exists n a > \p\ D = 1 such that for 
all n > n a , P/(©^) < a. Let n > n a , and © n be the set of observed sequences of length 
n triggered by a run with prefix p and whose correctness proportion exceeds e: 

© n = {ff G S" | 3p' £ SR n , p < p' A V(p') = a A CorP(cr) > e) . 

Let us prove that P(© n ) < a. On the one hand, since P/(© n ) > P f(E) = 1, Pj(© n n 
(£” \ ©n)) = 0. On the other hand, since P/(©^) < a, P/(© n O ©„) < P/(©^) < a. 
Thus, P/(©„) = P/(© n n© n )+P/(© n n(S q \©n)) < a. Because a was taken arbitrary, 
we obtain that limj^oc Py(© n ) = 0. Observe now that 

P ({p G SR„ | p ■< p' A CorP(TV)) > £}) = ^P/(6»). 

Therefore, lim^^oo P({// £ SR„ | p A p' A CorP('P( / o / )) > e}) = 0. In conclusion A is 
AFF-diagnosable. □ 


This characterisation shows that, for initial-fault pLTS, AFF-diagnosability can be 
reduced to the distance 1 problem. As one can perform the closure w.r.t. unobservable 
events and check the distance 1 in polynomial time, AFF-diagnosability for initial-fault 
pLTS belongs to PTIME. 


Example 4.11. Consider the initial-fault pLTS of Figure f.ll (qo,A^,A c ) and the 
prospect E = {cr £ | lim sup ngN \TSrh. > I}. j\ s a ‘fr’ } las a probability 3/4 to be 

observed at each step in Af and 1/4 in A c , P f(E) = 1 and P c (2£) = 0 where P f and 
P c are the probability measures of A? and A c . Therefore this initial-fault pLTS is AFF- 
diagnosable. In fact, as this pLTS has a single minimal faulty run, it is even uniformly 
AFF-diagnosable. 


Remark 4.1. There exists a single minimal faulty rim in every initial-fault pLTS. As 
a consequence, AFF-diagnosability and uniform AFF-diagnosability are equivalent for 
initial-fault pLTS. 


In order to understand why characterising AFF-diagnosability for general pLTS is 
more involved, consider the pLTS A presented in Figure 4.12 Recall that A is AFF- 
diagnosable as shown in the proof of Theorem |3.1[ page 
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Figure 4.12: An AFF-diagnosable pLTS where the distance 1 characterisation cannot 
be applied in a simple way. 


Let us look at the distance between pairs of a correct and a faulty states of A that 
can be reached by runs with the same observed sequence. On the one hand, we have 
d(A4(A qi ), M.(A qf )) < 3/16 since for any prospect E either (1) b u E E implying 
W M{Aq f \ E ) = 1 and F m ^o\E) > 13/16 or (2) b u </ E implying f> M{Aq f\E) = 0 and 
9 0 )(_g) < 3/16. On the other hand, d(M(A qc ), M(A qf )) = 1 since W >M( ' Aq f\b UJ ) = 
1 and pAl(Agc) (b u> ') = 0. 

Intuitively, the pair (qi,qg) is irrelevant, since the correct state q± does not belong to 
a BSCC of the pLTS, while (q c , qg) is relevant since q c belongs to a BSCC triggering a 
“recurrent” ambiguity. The next theorem characterises AFF-diagnosability for general 
pLTS, establishing the soundness of this intuition. 


Theorem 4.1. Let A be a pLTS. Then, A is AFF-diagnosable if and only if for every 
correct state q c belonging to a BSCC and every faulty state qg reachable by a faulty run pg 
suchthatq c is reachable by a run with same observed sequence, d{Ai(A qc ),A4.(A qf )) = 1. 


The proof of Theorem |4.1| due to its complexity and length, is divided into two 
lemmas, Lemma 4.2 and Lemma 4.3 given below, each of them stating one implication 
of the equivalence. 


Lemma 4.2. Let A be a pLTS. If there exists q c E Q c belonging to a BSCC, qg E Qg 

such that d(A4(A qf ),M(Aq c )) < 1 and runs qo q c and qo => qg such that V(p c ) = 
V(pg), then A is not AFF-diagnosable. 

This lemma is the easiest of the two. It is proved by contraposition. Assume there 
exist two states in A, q c E Q c belonging to a BSCC and qg E Qg reachable resp. by p c 
and pg with V(p c ) = V(pg), and with d(A4(A qc ), A4(A qf )) < 1. Applying Lemma 4.1 
to the initial-fault pLTS A 1 = {q' 0 , A qf , A qc ) where q' 0 is a new state, one deduces that 
A' is not AFF-diagnosable. First we relate the probabilities of runs in A and A!. Then 
we show that considering the additional faulty runs with same observed sequence as pg 
does not make A AFF-diagnosable. 


Proof. Let A be a pLTS, assume there exists q c E Q c belonging to a BSCC, qg E Qg such 

that d(A4(A qf ), A4(Aq c )) < 1 and runs qo q c and qo ^4 qg such that V(p c ) = V(pg). 
Let us introduce some notations: 


:= V{pg) = V(p c ), pg := P A(Pf), Pc ■= P a(Pc) ■ 
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Let p g (> pf) be the probability of the faulty runs with observed sequence oq: 

Pg = ^a{{p G SR W | V(p) = cr 0 , and p is faulty}) . 

For all n > |<r|, let & n be the set of observed sequences of length n “extending” pf. 

6 n = {cr £ £” | 3p £ SR n ,Pf ^ p A 'P(p) = cr} . 

Given a £ & n , we refine Pfp c and p g as follows. 

• Pf = IPU({P e SR.„ | Pf ■< p/\V(p) = cr}); 

• Pc = ¥ a({p G SR n I Pc ^ P A V{p) = cr}); 

• Pg = P_4({p £ SR n | p is faulty and V(p) = cr}). 

We introduce the initial-fault pLTS A! = Wo, A qf , A qc ) for some new state q' 0 . It is 

well-dehned since q c belongs to a BSCC so that A qc does not trigger faults. We write P 7 

there exist positive reals a',e' < 1 such that for all no 6 N there exists n > no- 


for the probability measure in A!. Since d(M{A qf ), M.{A q< ,)) < 1, due to Lemma 


4.1 


P_4'{,o £ SR n | q' 0 fq f < p A CorP (V{p)) > e} > a' 
This entails the following inequality for A: 


£ SR„ | Pf ^ p A 


P V C [P) 


P(p) _L PcAPiP) 


Pc 


+ »Pf 


> e'}) > 2 pja' . 


Indeed in A the probability of the set of faulty (resp. correct) run with observed 

p(p) 

sequence V(p) is (resp. p ^ p ): the probability in A! to go in qf (resp. q c ) initially, 
1/2, times the probability in A of the runs extending pf (resp. p c ) with observation V(p), 
Pf^ (resp. p£^), divided by the probability of pf (resp. p c ), pp (resp. p c ). Finally 
the 2 pf factor of the lower bound takes into account the fact that the probability of 
reaching qf in A' is 1/2 while the probability of p in A is pf. 

Observe that p n/ + c ^ (p) > £ is equivalent to v/ P ) + v (P ) > £ ' Pe+ (w )p/ • So defining 
G4EG- < 1 and a = ‘Ipfof < 2, the previous inequality can be rewritten: 


e = 


S'p c + (l-£')pf 


P({p £ SR n \ pf < pf\ 


p V c {P) 


P^ p) +P v f {p) 


> e}) > a 


Let & n be the subset of observed sequences of & n whose correctness proportion is 
greater than e when only considering extensions of pf, but smaller than e* = ^ when 
considering all faulty runs: 


6 n = W G & r . 


Pc +P°f 


> £ A 


Pc 


Pc +P°n 


< £*}. 
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< ~fPc and Pc < i^Pg- Therefore p a f < 

X > rr . ( 1 —g)g* 


Let a G & n , Pj ^ — y c mu p c ^ pzyyFg ■ j-uciciuic Vp ^ 
Summing over all sequences of & n : Eo-e 6 , p a j < 

Since p g < 1: E ctS 6 ^/ < < f ■ 

Thus, 


IP({p £ SR n 
IP({p G SR n 


v V[p) 

Pf<pA^—p^>e*})> 

Pc + Pg 

p^ CT ' _ a 

pf ~ pA n n P ) + n n P ) >£ »- E p / >a "2 


a 

2 ' 


Observe that given a G 6 n , CorP (a) > - , since we ignore correct runs p with 

Pc ~^~Pg 

V(p) = & that do no extend p c . So defining e = e* and a = d/2, for all no G N there 
exists n>ny. 


P V C {P) 

e SRn I Pf 1 P A fgp) - Wpi >£})>« • 

Pc + P 9 

Let po be the minimal faulty run such that po A pp. We observe that Cyl(pj) C Cyl(po)j 
so that 

P^ p) 


G SR„ | po A P A - WTp) > £ i) > a 

Pc + Pg 


which establishes that A is not AFF-diagnosable. 


□ 


Lemma 4.3. Let A be a pLTS. If for all q q q c and qo => qp with V(p c ) = V(pp), 
qp G Qp and q c G Q c belonging to a BSCC, d(A4(A qc ),A4(A qf )) = 1, then A is AFF- 
diagnosable. 

Let po be a minimal faulty run, a > 0, e > 0, oq = V(po) and no = |oo|. Before 
developing the proof, we sketch its structure and illustrate it in Figure 4.13 First, we 


extend the runs with observed sequence do by n\, observable events where n& is chosen 
in order to get a high probability that the runs end in a BSCC. 

Let <J G E™ 6 be such an observed sequence. We partition the possible runs with observed 
sequence oqg into three sets 91)/, 91)/, and 91/. 91/ is the subset of faulty runs while 
91/ (resp. 91/) is the set of correct runs ending (resp. not ending) in a BSCC. At first, 
we do not take into account the transient runs in 91/. We apply Lemma 4.1 to obtain 


an integer n a such that from 91/ and 91)/, we can diagnose with (appropriate) high 
probability and low correctness proportion after n a observations. Among the runs that 
trigger diagnosable observed sequences, some will exceed the correctness proportion, s, 
when taking into account the runs from 9!;/. Yet we show that the probability of such 
runs is small when cumulated over all extensions a leading to the required upper bound 


a. 
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ni 


high proportion 
of faulty runs 


n 0 n b 


Figure 4.13: Illustration of the proof of Lemma 4.3 


Proof. Let po be a minimal faulty run, a > 0, s > 0, gq = V(po) and no = |<to|- Since 
almost surely a random run ends in a BSCC, there exists n b such that for p = ^ 

P{p G SR no + n6 | do ^ 'P(p) A last(p) does not belong to a BSCC} < p . 


Let & = {a £ \ 3p £ 

• ^<t = \.P ^ SR no+nh 

• = {p ^ SR no+ni) 

• = I/ 5 ^ SR no+ni) 


SR no _|_ ni) po r< p A P(p) = (Joe}- Pick some a £ & and define: 

| P(p) = ct 0 cj A last(p) G Q/}; 

| iP(/9) = doer A last(p) G Q c and belongs to a BSCC}; 

| V{p) = <7o(JAlast( / o) G Qc and does not belong to a BSCC}. 


Temporarily, we ignore the runs of 94^. Let = {last(p) | p £ 94^} and Q} = 
{last(p) | p £ 94^}. For every pair ( qf,q c ) G Q} x Qf. consider the initial-fault pLTS 
A! = (q' Q , A qf , Aq c ) for some new state q' 0 , and denote P' its associated probability 
measure. Due to Lemma |4.l[ for all a 1 > 0, s' > 0, there exists n qf „ c such that for all 


n > n q f ,qA 


F '{P G SR n | q' 0 fq f <p A 


P™ 


Pi 


+ p' V f 


> s'} < o! 


where p (resp. p'^^) is the probability in A! of a correct (resp. faulty) run with 
observed sequence V(p). 
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Define in A. p^^ (resp. p to be the probability of a correct (resp. faulty) run 
with observed sequence V{p), Pf = min(P(p) | p G 91^) and p c = p (/°)- By a 

worst-case reasoning, one gets p'^^ > yPc 0<T and p'^^ < yjpf^^. Thus for all 
n > n 0 + n b + max(n 9/)?c ): 

V( P ) 

P{p G SR n I V e R* A p' ^ P A . ' Pc > e'} < 2a 

Pc " + 


where the factor 2 takes into account the first transition in A!. 

Choosing e' = + e ^ -e) P an< ^ a ' = 4 |e 5 after algebraic operations the previous 

inequality can be rewritten: 


P{p GSR n 


3p G R# A p' •< p A 


/ (p) 


/ (p) 


+^/ (p) 




a 

2l© 


Let n CT = no + rib + max(n g/i q c | (qq, q c ) G Qj X <2£) and ni = rnax(n 0 - a G 6) and 
consider n > n\. Ignoring the runs of 91^, one could detect the fault done in po with 
good accuracy and high probability n\ steps after it occurred. 

We now take into account the runs of 9lJ. Let p G {p G SR n | zip' G 91^ A p' < p}. 
Dehne p to be the probability of runs (1) with observed sequence V(p) and (2) 
extending runs of 9lJ. Since a correct run with observed sequence V(p) must have a 
prefix in 91^ or in 91^: 


CorP (V(p)) < 


p V c (p) +P^ p) 

P? (p) + pf (p) + Pf ^ 


Consider the following set of runs: 


91” = {p G SR„ | 3p G 9lf A p < p A 


pf (p) +pf (p) 


Pc (p) + pf (p) 


+P/ (P) 


> £ A 


pf (P) 


P(p) 

Pt 


+ 


£ 

f(p} - v 


Pf 


~ 'P(p) 

For p E one gets by algebraic operations, — > Pf • 

Thus P(fK”) < and Eaee P (^) < 2E " e ® P(;H F 

Due to the choice of nb, Eo-e6 p (9lf) < r l- and we derive Eo-e6 p (^1ct) < gr = § • 
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Summarising for all n > ny. 


e SR n | Po dt P A CorP(P(p)) > e} 

X: ^1/° e SR n I po A p A (Too- A P(p) A CorP^fi?)) > e} 


re6 


< X] ^{/° e SR n | 3p' e 91^ A p ^ p A 




> ol 


o-e6 


2 


P(p) A(p) . V(p) 

+P{p € SR n I Ve^Ap'SpA * ^ < i A * v(n , > 8 } 


p + p ^ ^ 2 p^^ + Pt ^+ p v f ^ 


Pc -r Pt 


■Pf 


. ol a 

< 6 —- + - = a 

'2 6 2 


which establishes the AFF-diagnosability of A. 


□ 


As an alternative to the proof of Theorem 4.1 one could mimic the approach by 


Kiefer and Sistla |KS16| for monitorability. The idea would be, from a pLTS A to 
derive two hidden Markov chains, say 7f c and ~Hf representing respectively the observed 
sequences for correct and faulty runs of A. However, to establish that distinguishability 
of die and Jif corresponds to AFF-diagnosability essentially relies on the same arguments 
we used in the above proof (and so this alternative approach would not simplify it). 
The difficulty lies in that the properties one conditions by to obtain T~L C and T~Lf, namely 
always correct or eventually faulty, anticipate on the future behaviour of the system; in 
contrast, the correctness proportion appearing in the definition of AFF-diagnosability 
only reasons about the possible behaviours up to the last observation. 


2 Verification of the diagnosability 

We now study the decidability of the different diagnosability notions for finite pLTS and 
in the positive case provide the complexity. The characterisations given in Section [l] 
play an important role in this study. Indeed, when a simple characterisation exists, 
the diagnosability problem is decidable (an algorithm consists in checking this charac¬ 
terisation). Conversely, when we did not exhibit a characterisation, we show that the 
problem is undecidable. 

2.1 Decidability results and upper bounds 

We start by showing how to check the characterisations defined in Section [lj therefore 
providing upper bounds to some of the diagnosability problems. We first consider exact 
diagnosability notions, and establish that they can all be solved in PS PACE. In all cases, 
to obtain the PS PACE upper-bound, we avoid building explicitly the exponential size 
product pLTS (that is used in the characterisations) and only explore it on-the-fly. 

The three results have similar proofs. As a consequence we first develop the case of 
FF-diagnosability then we simultaneously deal with both FA and lA-diagnosability. 
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Proposition 4.7. The FF -diagnosability problem is decidable in PSPACE. 


The proof consists in designing a PSPACE algorithm to check the characterisation 

This algorithm exploits Savitch’s Theorem [Sav70| which 
NPSPACE. This theorem allow us to use non-determinism 


given in Proposition 4.2 


establishes that PSPACE 
in our decision procedure. 


Proof. To obtain a PSPACE algorithm, we cannot build explicitly the product pLTS 
ylFF, which is exponential in the size of A. Given two states s,s' of -4 ff ; one can 
check in polynomial space in the size of A whether s' can be reached from s. Indeed, 
reachability of a state is known to be in non-deterministic logarithmic space in the 
size of the system, thus here NPSPACE, which is equal to PSPACE thanks to Savitch’s 
Theorem. Using this procedure, we can check whether a state s is not in a BSCC by 
guessing another state s' such that s' is reachable from s but s is not reachable from s'. 
Here again we apply Savitch’s Theorem. 

Thus the procedure that decides whether A is not FF-diagnosable consists in guessing 
a state s = ( q , U) with q € Q / and U ^ 0, checking that it is reachable from so and 
whether s belongs to a BSCC. □ 


We state below similar results for FA and lA-diagnosability problems. 
Proposition 4.8. The FA- and IA- diagnosability problems are decidable in PSPACE. 


The two proofs are similar to the proof of Proposition 4.7 we design a decision pro¬ 


cedure which uses non-determinism to check the characterisation given in the previous 
section and the non-determinism is removed using Savitch’s Theorem. 


Proof. We first check the characterisation of FA-diagnosability given in Proposition 4.4 
without explicitly building the product pLTS »4 fa- First given a state (q, U, V ) of „4 ,fa 
we can check in polynomial space whether it belongs to a BSCC (as in the proof of 
Proposition |4.7[ ). We can also check in polynomial space whether some state (q ', U', V') 
with U' = 0 or V' = 0 can be reached from (q, U, V) by guessing such a state and 
then checking the reachability condition. Combining the two, this provides a polyno¬ 
mial space algorithm to check whether (q, U, V ) belongs to a BSCC in which no state 
{q ', U', V') fulfils U' 0 and V' 0. Thus the procedure that decides whether A is not 
FA-diagnosable consists in guessing a state s = ( q,U,V ), checking that it is reachable 
from sq and belongs to a BSCC where all states (q', U', V') fulfil U' 7 ^ 0 and V' / 0. 


We use the characterisation of lA-diagnosability given in Proposition 4.5 without 
building explicitly the product pLTS „4 |a- First, given a state (q,U,V,W) of „4 |a, we 
can check in polynomial space that it belongs to a BSCC (as in the proof of Propo¬ 
sition 4.7). We can also check in polynomial space whether it is coreachable from a 
state (q', U\ V', W’) that fulfils U' = 0 or W' = 0 by guessing such a state. Combining 
the two procedures, we can check in polynomial space whether (. q , U, V, W ) belongs to 
a BSCC where all states (q\ U\ V', W’) of the BSCC fulfil U' / 0 and W’ / 0. Thus 
the procedure that decides whether A is not lA-diagnosable consists in guessing a state 
s = ( q , U , V, W), checking that it is reachable from the initial state So and belongs to a 
BSCC where all states (q', U 1 , V ', W') fulfil U' / 0 and W' / 0. □ 
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For approximate diagnosability, we focus on AFF-diagnosability and establish a com¬ 
plexity upper-bound, relying on the characterisation from the previous section. 

Theorem 4.2. The AFF -diagnosability problem is decidable in PTIME for pLTS. 


Proof. The decidability and complexity results rely on the characterisation of AFF- 
diagnosability showed in Theorem 4.1 Reachability of a pair of states with the same 
observed sequence is decidable in NLOGSPACE by an appropriate “self-synchronised 
product” of the pLTS that we detail below. Since there are at most a quadratic number 
of pairs to check, and given that the distance 1 problem can be decided in polynomial 


time due to |CK14| (as recalled in Proposition |4.6[ ), the decidability and PTIME upper- 
bound follow. 

We now define the appropriate “self-synchronised product” of the pLTS mentioned 
above. Given a pLTS A = (Q, go, £, T, P) we build the product LTS A <S> A = (Q x 
Q,{Qo,qo},^,T') where ((gi,g 2 ),a, {<&,<&)) ^ T ' if 

• if a E T, u then either (gi, a, q\ ) E T and q 2 = g 2 or (g 2 , a, q' 2 ) E T and q[ = qi ; 

• else (a E £ 0 ), (gi,a, g^) E T and (g 2 ,a, g^) E T. 


A pair of state (g, q r ) is reachable in A <8> A from (go, go) if and only if there exist two 
runs p and p' of A such that last(/o) = g, last( / o / ) = q' and V{p) = V(p'). Therefore, A 
is AFF-diagnosable if for any pair of states reachable in A® A, (g, q'), with g correct, 
belonging to a BSCC of A. and q' faulty, then d(Ai(A q ), Ai(A q /)) = 1. A11 tests can 
be checked in PTIME. □ 


We thus have decidability of every notions of exact diagnosability and of one notion 
of approximate diagnosability, AFF-diagnosability. Surprisingly, AFF-diagnosability, 
which definition seems more complicated as it depends on the exact probabilistic values 
of the transitions contrary to the definitions of the exact notions of diagnosability, has 
a lower upper bound. 


2.2 Hardness of Diagnosability 


We gave upper bounds on the complexity of diagnosability in Subsection 2.1 We 


now provide tight lower bounds: on the one hand we establish undecidability of the 
approximate diagnosability notions that were not characterised, and on the other hand 
we provide a PS PACE lower bound for the exact diagnosis. 


2.2.1 Undecidability results 

After having previously proved that AFF-diagnosability can be solved in polynomial 
time, we now establish that all other specifications of approximate diagnosability are 
undecidable. This result could be expected for eFF-diagnosability and uniform eFF- 
diagnosability since it is often the case for problems mixing probabilities, partial ob- 
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servation and quantitative requirement (here represented by e)|^] On the contrary, the 
undecidability of the uniform AFF-diagnosability problem is at first sight surprising 
since it is a slight variation of the AFF-diagnosability problem. In fact the reduction 
for the latter problem is more intricate than the one for the eFF- and uniform eFF- 
diagnosability. We reduce the emptiness problem for probabilistic automata [Paz m 
to both problems. Let us first details this problem. A probabilistic automaton is an 
automaton enhanced with probabilities on the transitions so that given a state and a 
letter, the sum of the probabilities of the transition exiting each state and labelled by 
the given letter is 1 . 


Example 4.12. Figure f-lf represents a probabilistic automaton which initial state is 
qo and set of accepting states is {( 72 }- The sum of the probability of the transitions 
exiting qo is 2 as there is a transition labelled by a ‘a’ and a transition labelled by a ‘b 
The word bob has probability 1 /2 to end in c /2 an d 1 /2 to end in q\. 



a, 1/2 


Figure 4.14: An example of probabilistic automaton. 


Definition 4.6. A probabilistic automaton is a tuple A = (Q , do, E, (P a ) ag v, F) where 

• Q is a set of states with FAQ, the set of final states; 

• Sq € Dist(Q) is the initial distribution; 

• E is an alphabet; 

• For every a E E, P a is a stochastic Q x Q matrix. 

When P a [q,q'\ > 0, there is a transition from q to q' labelled by a and P a [q,q']. 
Given a word w = a\...a n E E*, the acceptance probability of w, P^tn) is defined 
by P a(w) = J2 qo ^Q^o{qo)J2q^F F ^[ < lo,Q\ where P„, = P ai •■■P a „. Given a rational 
threshold 0 < e < 1, the language Ca,e is defined by Ca,e = {w E E* | P^ru) > e}. 
For a probabilistic automaton A and a threshold e, the emptiness problem asks whether 

4 one of the most famous example is the undecidability of the emptiness problem for probabilistic 
automaton |Paz71| that we detail below, see also |MHC03| for some examples of undecidable problems 
for partially observable Markov decision process, a formalism which also contains a form of control of 
the system 
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£-A,e = 0- This problem is undecidable even for a fixed 0 < e < 1 |Paz71j . The problem 
is also undecidable for the language defined by {re E E* | > e}, i.e. when the 

inequality is not strict. One can also restrict oneself to automata such that every word 
w satisfies 1/4 < P^u;) <3/4. This can be ensured by a simple construction. Given 
a probabilistic automaton A with initial distribution So and a threshold 0 < A < 1 , we 
build the probabilistic automaton A' that contains the states of A and two additional 
states q a and q r . The new initial distribution goes in q a and q r with probability 1/4 and 
with probability 1/2 it uses the initial distribution of A. In the last case, the behaviour 
is the one of A. from q r and q a everything can be observed with a self-loop and q a is a 
final state. For every word w, we have 


P.4'0) 


p a(w) 1 
2 + 4' 


This is the sum of the probability to be accepted after starting initially in the A compo¬ 
nent of A 1 plus the probability to go in q a . As a consequence, every word is accepted in 
A' with probability between 1/4 and 3/4 and a word is above the A threshold in A iff it 
is above 1/4 + A/2 in A'. Thus the emptiness is also undecidable with this restriction. 
Note that this assumption relies on the use of an initial distribution do instead of an 
initial state qo . Indeed, with an initial state, the word e would have probability 0 or 
1. When this assumption is not needed, we use an initial state instead of an initial 
distribution. Another important undecidable problem that is used in a latter chapter is 
the value 1 problem. It asks for a probabilistic automaton A if for all e > 0 Caa-e A 0- 
In other words, does there exists words of arbitrarily high probability? This problem is 
known to be undecidable |GQ10| . 


Theorem 4.3. For any rational 0 < e < 1, the eFF -diagnosability and uniform eFF- 
diagnosability problems are undecidable for pLTS. 


We make a reduction from the emptiness problem of probabilistic automata. From 
a given probabilistic automaton A = (Q , do ) (Pa)aeS , F ), we build a pLTS which 
produces with probability 1 runs whose observed sequence belongs to £*$“ (where 
ft 0 E) and for all n > 2, the correctness proportion CorP of w\\ n , with i« G S*, satisfies 
CorP(tc(J n ) = P_ 4 (w). In other words, if a word w is accepted with probability greater 
than e, then the ambiguity of the word u>(j 2 is greater than e and every faulty run with 
this observation will remain ambiguous. 

Proof. Let A = ( Q, So, E, (Pa)aes> T) be a probabilistic automaton. W.l.o.g we assume 
1/4 < Pa(w) < 3/4 for every w E E*. Define the pLTS A = (Q 1 , qo, S', T', P') as 
follows: 


. £' = EW{#,f,u}, H' u = {f,uy, 

• Q' = Q^{qo,ql,q i f ,f^}\ 

• T' = {(q 0 ,u,q) \ q E Q,5 0 (q) > 0} U {(<?, a, <?) | q, q' E Q,a E E,P a [q,<f] > 
0} U {(q, (t, ql | q E F} U {(q, jj, qj \q E Q \ F} U {ql, jj, q\} U { q f, /**} U {/ # , jj, / # } 







116 


Algorithmic analysis of the diagnosability of finite pLTS 


• P' is defined by: 


— For all q £ Q such that 5o(q) > 0, P'(qo,u,q) = 4'o(g); 

— For all q e Q and a 6 X, P '(q, a, q') = 

- For all q <E F, P'(<?, Jt, ql) = 5 

- For all q € Q \ F, P'(g, (t, 5 /) = 

- P'(g#, f, /#) = P '(/*, (t, /") = P'g, ft, g2) = 1 . 

This reduction is illustrated in Figure |4.15| In each state, the sum of the probabilities 
of the exiting transitions correctly sum to 1. For instance, let q £ F and a £ X, then 
Eg' £ Q P «|!.9l = !. thus: 


X] p, (9>«,ff 7 ) = X X Tdilr = 

(g,a,g')eT' a£S g'eQ 


+ 


1 + lXl 1 + IX 


= 1. 



Figure 4.15: From probabilistic automata to pLTS. 


We claim that the following three assertions are equivalent: 

1 . A is eFF-diagnosable; 

2 . A is uniformly eFF-diagnosable; 

3. C A ,e = 0- 

Given that uniform eFF-diagnosability entails eFF-diagnosability, we only show that 
item 1 implies item 3, and item 3 implies item 2. The first implication is proved by 
contraposition. 

1 implies 3 Assume that there exists a word w 6 X* such that Px(w) > e. Consider 
the set of signalling correct runs with observed sequence w\\ n+2 . By construction, 
its probability is . Similarly, the set of signalling faulty runs with ob¬ 
served sequence w$ n+2 has probability • Thus CorP(u;t| n+2 ) = P^(u;) > 

e. By assumption on A, P a(uj) < 3/4 < 1, so that the set of faulty runs with 
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observed sequence w$ n+2 is non-empty. Pick p a minimal faulty run with observed 
sequence Using the above probability values, for every n > 0: 

nW G SR n+Ho | p A p' A CorP(P(p')) > e}) = P (p) . 

Thus A is not eFF-diagnosable. 

3 implies 2 Assume that for every word w € £* , Pr^ (vj) < e. Let p be a minimal 
faulty run of A. By construction, its observed sequence is of the form rcfj 2 with 
w £ E*. Using the same reasoning as above, for every p A p'\ CorP (V(p')) = 
Pf/i (vj). and thus CorP (V(p')) < £. Therefore, for any a > 0, choosing n a = 0, 
one gets: 

P({p' € SR na+ | p | | p A p' A CorP(P(p')) > e}) = 0 . 

So A is uniformly eFF-diagnosable. 

This completes the proof that eFF-diagnosability and uniform eFF-diagnosability 
are undecidable. □ 

Uniform AFF-diagnosability is also shown to be undecidable by a reduction from the 
emptiness problem for probabilistic automata. 

Theorem 4.4. The uniform AFF -diagnosability problem is undecidable for pLTS. 

As this reduction is more involved, we start by a developed sketch of proof and 
then give the full proof. We proceed by a reduction from the emptiness problem for 
probabilistic automata where w.l.o.g. one assumes that the acceptance probability of 
any word lies between 1/4 and 3/4. Given such a probabilistic automaton one builds a 
pLTS as follows. 

• With probability 1/2 one enters one of the two copies of the automaton whose 
probabilities are modified in a similar way as in the previous proof. 

• In a non-final (resp. final) state of the first (resp. second) copy, one may exit the 
copy of the automaton by taking a transition labelled by b (resp. f) and enter a 
terminating state. In a final state (resp. non-final) state of the first (resp. second) 
copy, one may “restart” the copy of the automaton by taking a transition labelled 
by ft which lead to the initial state of the copy. 

• The terminating state of the first copy iteratively outputs with probability 1/2 jj 
or b while the terminating block of the second copy endlessly outputs b. 

Due to the behaviour of the terminating blocks, the correctness proportion of a faulty 
run goes to 0 as its length increases. Thus the pLTS is AFF-diagnosable. The element 
that will depend on the probabilistic automaton is the uniformity of the convergence. 

Observe that the language of the observed sequences of minimal faulty runs extended 
by one transition is (S*jJ)*S*b. 
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Assume there exists a word w with acceptance probability strictly greater than 1/2. 
Then in A, the correctness proportion of (iuj}) n b fulfils: lim n ^.oo CorP((rejj) n b) = 1. 
Due to this property (and the behaviours of the terminating blocks), the pLTS is not 
uniformly AFF-diagnosable. If no such word exists, then for any w = toijj^jt • ■ • iCfcb, 
CorP(w;) < 3/4. Due to this property (and the behaviours of the terminating blocks), 
the pLTS is uniformly AFF-diagnosable. 

We now develop the full proof. 

Proof. Let A = (Q, 5q, X, (P a )aeS; F) be a probabilistic automaton such that the ac¬ 
ceptance probability of any word lies between 1/4 and 3/4. Define the pLTS A = 
(Q ', q' 0 , S', T', P') as follows. 

• S' = S l±l {tt, b, u, f}, S' uo = {u, f}; 

• Q' = {q u ,q f \qeQ}uW 0 ,b u M}; 


T ' = {(q'o,u,q u ) | 5 0 (q) > 0}\J{(q' o ,u,q f ) \ 8 0 (q) > 0} U {(<?“, (t, <?o) I Q G F ,^o(q) > 0} 
U {(q f , lq f )\q€Q\F , 5 0 (q) > 0} U {(<*“, b, b u ) \ q G Q \ F} 

U {b u , ft, b u } U {b u , b, b u } U {b f , b, b f } U {(q f , f, b f ) \ q <E F} 

U {( q u ,a,q’ u ), (q f ,a,q f ) \q,q' &Q, a <E S,P a [q,q'} > 0} 


P' is defined by: 


— For 

all 


Q with 5 0 (q) > 0), P '(q' Q , 

u,q u ) = 1 

— For 

all 

(q u , 

a,q ,u ) G T', P '(q u ,a,q ,u ) 

_ P a [«,?'] 

' 1 +|£| 

— For 

all 

( q f , 

a,q lf )er, P'(qf,a,q'f) 

_ P a[q,q'] 
~ 1+|E| 

— For 

all 

(q u , 

#,?&)€ T', P ''(?“, tt,9g) = 

<5o(go). 
1 +1=1 ’ 

— For 

all 

(< q f , 

U)^, P'(q f ,lq f 0 ) = 

<5o(?o). 
1 +1=1 ’ 

— For 

all 

(q u , 

b ,b u ) g T, P'(g“, b, b u ) = 

1 . 

1+1 = 1 ’ 

— For 

all 

( q f , 

f ,b f )&r, PV,f,6/) = 

1 . 

■ 1 +|E| ’ 

- P'(6“,t 

>,b u ) 

= P'(6«,tt,6«) = 



P'(^,b,6^) = 1. 


This reduction is illustrated in Figure 4.16 In each state, the sum of the probabilities 
of the exiting transitions correctly sum to 1. For instance, let q E Q, 


S pV,», 9') = £ S TTfir 

(F ,a,q')eT' aG^q'GQ 1 1 


1 


1 


1+S 1+E 


= 1 . 
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Figure 4.16: From probabilistic automata to pLTS: rectangles surround the two copies 
of the state space of the probabilistic automaton. 


We claim that A is uniformly AFF-diagnosable if and only if Ca,i/ 2 = 0. 

Observe first that for all q € Q, C^iAqf ) C C u {A q ^) so that all faulty runs are ambigu¬ 
ous. 

• Assume that there exists a word w G E* such that > 1 / 2 . We prove that A is 

not uniformly i^FF-diagnosable. So we pick arbitrary 0 < a < 1 and n a . 

Consider the observed sequence a n = (w$) n b for some n to be fixed later. As every 
word is accepted with positive probability by A, it is ambiguous. Let 

_ P({// £ C | P(p') = a n }) 
ln P({p' G F | V(p') = a n }) ' 

Since Pa(w) > 1/2, y n fulfils linx )WOO = oo. 

Let p n be a minimal faulty run with V(p n ) = a n . Let p be a signalling run extending 
p n with \p\ Q = \p n \ 0 + n a . Then V(p) = a n \> na . By a straightforward examination of A 
one gets: 

p(jy € c I V(p’) = <r n b»°}) = ln 2~ n <* 

P({p' G F I V(p') = (J n b na }) 1 +7 n 2 _n “ 

Choosing n such that 'y n 2~ n °‘ > 1, one gets: CorP(p) > 1/2. So: 

P({p € SR„ a+ |p n | o | p n ■< p A CorP(P(p)) > *}) = P (p) > aP(p) . 

Thus A is not uniformly ^FF-diagnosable. 
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• Conversely assume that for every word w G £*, Pa(^) < 1/2. Combining this 
assumption with the hypothesis that P^tc) >1/4, one deduces that for every observed 
sequence o G (X U {fj})*b, CorP(cr) < 3/4. On the other hand, for every minimal faulty 
run p, V(p) G (SU{#})*b. 

Pick any positive e, a and consider an arbitrary minimal faulty run p. The observed 
sequence o' of a faulty run p' that extends p fulfils o' = V(p)\> n for some n. After a 
new occurrence of b the fraction between the probability of correct runs with observed 
sequence o' b over the probability of faulty runs with observed sequence o' b is halved. 
Thus choosing n a such that < s, for all n> n a : 

P({p' G SR n+ | p | o | p A pi A CorP (P(p)) <£)= P(/») > (1 - a)F(p) . 

Thus A is uniformly e F F-diagnosable and since e was chosen arbitrarily, A is uniformly 
AFF-diagnosable. □ 


The undecidability of (uniform) eFF- and of uniform AFF-diagnosability are due 
to different reasons. For (uniform) eFF-diagnosability, it is mainly caused by the use 
of a quantitative requirement (shown through the use of e). For the uniform AFF- 
diagnosability, the problem arises as the detection speed of the fault strongly depends 
on the behaviour of the system before the occurrence of the fault. Limiting the behaviour 
of the system before the occurrence of a fault can raise decidability results (as in initial- 
fault pLTS where uniform AFF-diagnosability is diagnosable as it is equivalent to AFF- 
diagnosability) . 

Thanks to Proposition 3.5 page ITol and the definition of AFF-diagnosability, we 


know that a system is AFF-diagnosable iff for all £ > 0 there exists an eFF-diagnoser of 
the system. Limiting oneself to finite memory can sometimes bring better decidability 
or complexity results (in [BFH^lTj for example), ft is therefore natural to question if 
such a restriction would make uniform AFF-diagnosability decidable. There is several 
ways to add the finite-memory restriction to AFF-diagnosability: 


1. a system is AFF-diagnosable with finite memory if for all £ > 0 there exists a 
finite-memory £FF-diagnosers; 

2. a system is AFF-diagnosable with finite memory if there exists A > 0, such that 
for all 0 < £ < A there exists a finite-memory £FF-diagnosers; 

3. a system is AFF-diagnosable with finite memory if there exists a sequence (£ n )neN 

such that Vn, £ n > 0, £ n 0 and for all n G N there exists a finite-memory 

£ n -diagnosers; 


These three notions are however equivalent since if D is a finite-memory £FF-diagnosers 
for some £ > 0, then D is a finite-memory £'-diagnoser for all s' > e. Surprisingly, this 
restriction complexifies the problem as stated in the following proposition. 


Proposition 4.9. The AFF -diagnosability with finite memory problem is undecidable 
for pLTS. 
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The proof is obtained thanks to the reduction made in the proof of Theorem 4.4 


Indeed, in this particular case, we can show that the uniform property is equivalent to 
the finite-memory restriction. 


Proof. Given a probabilistic automaton A, let A be the pLTS built in the reduction of 

We know that A is uniformly AFF-diagnosable if and only 
We show that A is uniformly AFF-diagnosable iff for every e > 0 there 


the proof of Theorem |4.4 
if C 


-A, 1/2 — 

exists a finite-memory eFF-diagnoser of A. which establishes the undecidability. 

Assume that A is uniformly AFF-diagnosable. Let e > 0. By definition of uniform 
AFF-diagnosability, there exists no £ N such that for all minimal faulty run p £ minF 
and all n > no, P(Cyl(p) n FAmb^ + i | o ) < If ^. As Cyl(p) is a single infinite run, this 
means that P(Cyl(p) H FAmb^, + i | o ) = 0. Let D be the diagnoser that counts how many 
b were observed and outputs T iff this number is above no and no W was observed after 
the first b. This diagnoser can be represented with finite memory as it only needs to 
count up to a fixed value no- Moreover it is an eFF-diagnoser: 


correctness, if D(w ) = T for some w £ X 0 U {b, j}}*, then there exists w' £ U {fj}* such that 
w = w' b n with n' > uq. As in A, every word is accepted with positive probability, 
there exists a minimal faulty run p with observation w'. Let pj be the infinite 
run of Cyl(p). By uniformity, pp fL FAmb^, + | p | . Moreover, the prefix of length 
n! + \p\o of pp has observation w. Thus CorP(u;) < e. 

reactivity. The n observations following a fault are b, and no jj can be observed after the first 
b in a faulty run. Thus, for all m > n, P (p 1 £ F n SR m | D(V(p)) =?) = 0 which 
implies reactivity. 


Conversely, assume that A is not uniformly AFF-diagnosable. There thus exists e > 0 
such that A is not uniformly eFF-diagnosable. Suppose there exists a finite-memory 
eFF-diagnoser with m memory states. As A is not uniformly eFF-diagnosable, there 
exists a minimal faulty run p such that for all n < m + 1, P(Cyl(p) H FAmb^i i ) > 0. 
Since there exists only one infinite run pp extending p, this means pj 6 FAmb^ + |^ o . 
Consider the observation V(p) b m+1 . The last m + 1 memory states visited in the finite- 
memory diagnoser while reading this observation are denoted s\,... s m +i- None of these 
memory state can claim a fault by correctness of the diagnoser and as pj £ FAmb^ + i . 
for n < m+1. Moreover, as the finite-memory diagnoser has m states, there exists 
i. j < m+1 such that s* = Sj. there thus exists a cycle labelled by a number of b in 
the finite-memory diagnoser. By determinism of the diagnoser, it means that for all 
n £ N, D(V{p) b m+1 ) =? which contradict the reactivity requirement. There thus does 
not exists a finite-memory eFF-diagnoser. □ 


2.2.2 PS PACE-hardness of exact diagnosability 

In order to establish a lower bound for the complexity of exact diagnosability, we intro¬ 
duce a variant of language universality. 
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Definition 4.7. A language C over an alphabet £ is said eventually universal if there 
exists a word ueS* such that v _1 C = £*. 

Several variants of the universality problem were shown to be PS PACE-complete 
in iRSXl 2| but, to the best of our knowledge, eventual universality has not been con¬ 
sidered. 

Because of our diagnosis framework, we focus on live non-deterministic finite au¬ 
tomata (NFA). Similarly to pLTS, an NFA is live if from every state there is at least 
one outgoing transition. The language of an NFA A, denoted £(A), is defined as the 
set of finite words that are accepted by A. 

Proposition 4.10. Let A be a live NFA where all states are terminal. Then deciding 
whether C(A) is eventually universal is PSPACE -hard. 

Proof. We reduce the universality problem for NFA, which is known to be PSPACE- 
complete [MS72] to the eventual universality problem. Let A = (Q, go, X, T, F) be an 
NFA. Starting from A, one builds in polynomial time the NFA A' = (Q' , go,X' ,T',Q') 
where £' = £ 0 {()}, Q 1 = Q W {s}, and 

T' = TU{(q, tt, go) I g € F} u {(s, a, s) I a E £} U {(g, a, s) \ a E £, g f> A } 

with g -f^ A meaning that there is no transition exiting q in A. The additional state s 
and the associated transitions are added to ensure that A! is live, they do not alter the 
accepted language. 

• Assume that C{A) = £*. Any word w over the alphabet £ ; can be decomposed into 
w = w lttw^tt ■ • • tt w n with Wi E £*. For each factor ng, since A is universal, there exists a 
run pi on Wi ending in some terminal state qi E F. Therefore w is accepted in A' by the 
run pifjp2tt • • • tt Pn- Hence A! is universal, and thus eventually universal: e _1 £(A / ) = £'*. 

• Conversely assume that A' is eventually universal and let v E X 7 * be such that 

v~ l C(A!) = £'*. Given w E £*, we consider the word w’ = ujtwjt. Since A! is eventually 
universal with witness v, vJ E T^A 1 ) and there exists an accepting run that can be 
decomposed as: pjtp^go- As a (J can only be read in a final state and leads to go, the 
run p', which corresponds to the word w, has go as initial state, ends in a final state of 
A, and by construction of A' only uses transitions of A. So p' is a run of A that accepts 
w. Therefore w E £(A), and A is universal. □ 

Now that we established that universal eventuality is PSPACE-hard, we can use it 
to establish a complexity lower bound for the different exact diagnosability problems. 

Proposition 4.11. The FF -diagnosability. FA -diagnosability and \/\-diagnosability prob¬ 
lems are PSPACE-hard. 


Proof. The proof is by reduction from the eventual universality problem. Let A be a live 
NFA over £, in which all states are final. One builds in polynomial time the initial-fault 
pLTS A = (q' 0 , A?, A c ) depicted in Figure 4.17 where £ 0 = £ l±J {Jj}, X u = {u, f} and all 


transitions outgoing a state have the same probability. A? consists of a single state on 
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which any letter of £ can be read with a self loop. A c is a copy of A to which we add a 
new state q§ to which one can access by a transition labelled by ft from any state of the 
copy of A. 

We establish the following two implications (note that they do not use the same 
diagnosability notion): 

• A is not FA-diagnosable implies A is eventually universal; 

• A is eventually universal implies A is not FF-diagnosable. 

Since FA-diagnosability implies lA-diagnosability, which implies I F-diagnosability which 
is equivalent to FF-diagnosability according to Theorem |3.1[ page |73[ this proves that 
all three notions are at least as hard as eventual universality. 



Figure 4.17: A reduction for PSPACE-hardness of IF-, FA- and lA-diagnosability. 


• Assume that A is not FA-diagnosable. By Proposition 4.4 
reachable BSCC C with some state s = ( q, U, V) 


either „4 fa contains a 
E C such that q E Qf and U A 0 or 
Af/\ contains a reachable BSCC C with some state s = ( q, U,V) E C such that q E Q c 
and V A 0. The latter case is excluded since the only correct state belonging to a BSCC 
of ylpA contains as first component and is only reachable by a transition labelled 
by ft. As this observation cannot occur in a faulty run, q = q$ implies V = 0. Consider 
the former case: obviously q = fo- Since C is a BSCC and /o is a sink state in A. for 
every state s' = (q 1 , U ', V) E C, one has q’ = /o and U' A Since in /q all events of £ 
are enabled, this implies that for all w E £*, there is a correct run p\ in A starting from 
some state of q E U with observed sequence w. Consider an observed sequence v E £* 
labelling a run in „4 fa from the initial state to s. Then there is correct run in A from 
q' 0 to q with observed sequence v. So the run p = poPi has vw as observed sequence. 
Since p = q' 0 up' with p 1 a run of A starting from qo, vw E C{A). This holds for any 
word w, thus v~ l C(A) = £* and A is eventually universal. 

• Assume that there exists a word v E £* such that v~ l C(A) = £*. Of course, any 
word extending v is also a witness that A is eventually universal. Let v' E £* be such 
that some faulty run with observed sequence vv' ends in a BSCC C of »4 ff- Since 
(vv')~ 1 C(A) = £*, all states of C are of the form (fo,U) with 17/0. Therefore, by 
Proposition 4.2, A is not FF-diagnosable. □ 


Since the lower bounds matches the upper bounds, the different notions of exact 
diagnosability are PS PACE-complete for finite pLTS. 
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3 Diagnoser construction 

When a system is shown to be diagnosable, the next step is to build a diagnoser. A 
diagnoser that works in every case would keep track of every run compatible with the 
observed sequence and give a verdict depending on the nature of this set of runs. This 
diagnoser, by nature, uses unbounded memory. For implementation purpose, we are 
rather interested in finite-memory diagnosers as defined in Section □ of Chapter [3] We 
explain how to automatically build a finite-memory diagnoser for a diagnosable system. 
This is not possible for every notion of diagnosability however. Indeed, we showed in 
Proposition |3.4[ page [69l that eFF-diagnosers may need infinite memory. Therefore we 
do not develop approximate diagnosability here, and focus on exact diagnosability. 

3.1 FF-diagnoser 

We start with FF-diagnosers. These diagnosers only provide information about faulty 
runs. In the sequel we fix A a finite pLTS. 

Proposition 4.12. If A is an FF -diagnosable pLTS with n correct states, one can build 
an FF-diagnoser for A with at most 2 n memory states. 

Proof. The idea of this proof and of all the following proofs for constructing a finite- 
memory diagnoser is to use the characterisation given in Section [I] For example, in order 
to construct the FF-diagnoser, we first build the FF-automaton of A. Then we define 
the finite-memory diagnoser on its structure. Finally we show that the constructed 
diagnoser is indeed an FF-diagnoser thanks to the FF-diagnosability of the system. 

For an FF-diagnosable pLTS A with FF(M) = (S,sq,A,F), its deterministic and 
complete FF-automaton, we define the finite memory diagnoser (5, X 0 , up, so, Df m ) with 
up(s, a) = s' if (s, a, s') G A and Df m (U) = T iff U = 0. Let us show that the induced 
diagnoser D is indeed an FF-diagnoser, and that it has at most 2 n memory states, where 
n is the number of correct states of A. 

commitment When U is empty, it remains empty forever which implies commitment. 

correctness When D outputs the verdict T, FF(M) is in the state associated with 
0. As U contains the set of correct states reachable with the current observed 
sequence, the observed sequence is surely faulty. 

reactivity If an infinite faulty run p is such that D(V(p)) =? then, by construction 
of FF(M) and definition of D , for every length n 6 N, there exists a finite correct 
signalling run p n G SR„ such that V(p n ) = V{p\ n ). Using Konig’s lemma, since 
A is finitely branching, one can extract an infinite correct run p^ such that 
'P(Poo) = P(p), so that p G FAmboo. Moreover P(FAmb oc ) = 0 as A is FF- 
diagnosable. Putting everything together, for every minimal faulty run p, P({p 7 G 

n\pPp'AD(V(p'))=7}) = 0. 

size The memory states are states of FF(M), which are themselves subsets of correct 
states of A. Therefore, D uses at most 2 n memory states, with n = \Q C \- 
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□ 


We now show that the size order of the previous FF-diagnoser is optimal. 

Proposition 4.13. There is a family {M n } ne N of FF-diagnosable pLTS such that A n 
has 2n + 2 correct states and it admits no FF-diagnoser with less than 2 n states. 



Figure 4.18: An FF-diagnosable pLTS requiring an FF-diagnoser with exponential size. 


Proof. This proof is inspired by a similar result of lower bounds on controllers es¬ 
tablished in [HHMS13] . Consider the example of Figure 4.18 where X D = {a, b, c} 
and the initial state is go- 


served sequence belongs to L = {a, b}*a{a, b} n l c + 


Consider a finite faulty run including a c event. Its ob- 

Since any finite correct run has 
an observed sequence belonging to £ = {a, b}* U {a, b}*b{a, b} n ~ 1 c + and C C £ = 0, 
FAmb ra t+J CAmb n C {p \ V(p) G {a, b} 71 }. Since linin^oo P({p | V(p) G {a, b} 71 }) = 0, the 
pLTS is FA-diagnosable and so lA-diagnosable and FF-diagnosable. 

Intuitively, when a c is observed, any FF-diagnoser must have remembered the ob¬ 
servable event that happened n steps earlier to know if the run is faulty or not. Thus, 
it must remember the last n observed events, in case a c event occurs. 

More formally, assume there exists a diagnoser D = (M, E, mo, up, Df m ) with less 
than 2 n memory states. Then there exist two distinct words w\ G {a, b} n and u >2 G 
{a, b} n leading to the same memory state: up(mo,ttq) = up(mo,u> 2 ). The words w\ 
and W2 differ at least from one letter say w\[i\ = b and = a. Consider for 

k > 1, the signalling correct run p\^ corresponding to observed sequence W\a l ~ l c k whose 
sequence of visited states is q^ri ... r k+1 and the signalling faulty run p 2 ) k corresponding 
to observed sequence W 2 d i ~ 1 c k whose sequence of visited states is q^loh ■ ■ ■ ^ +1 - They 
also lead to the same memory state. By correctness, D{w\a l ~ l c k ) =?. Thus for all 
suffixes p of p 2 ,i; D(p) ='■ contradicting the reactivity of D. □ 


3.2 FA-diagnoser 

We now turn to FA-diagnosability which not only considers the diagnosis of faults but 
also of correct runs. We build the FA-diagnoser from the FA-automaton similarly to 


what was done in Proposition 4.12 
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Proposition 4.14. A pLTS A is FA -diagnosable if it admits an FA -diagnoser. More¬ 
over, if A is a FA- diagnosable pLTS with n states, one can build an FA-diagnoser with 
at most 2 n memory states. 

Proof. Let A be a pLTS. Assume first that there exists an FA-diagnoser D for A. For 
every n E N, we define FD n = {p E Ft \ D(fP(pi n )) = T} the set of runs that are 
diagnosed faulty after n observed events, and symmetrically CD n = {p E Q | Vm > 
n, D(V(pirn)) = _L} the set of runs that are persistently diagnosed correct after n 
observed events. The sequences (CD n ) n eN and (FD n ) ng N are non-decreasing. As ? -4 _L 
and ? -< T, for every run p E Q, D[ n f(V(p)) =? is equivalent to p (J n (FD n U CD n ). 
Thus we have UneN^Dn U CD ra ) = {p E Fl \ D in f(V(p)) 7 ^?}. Since D is reactive, 
P({p E Fi | D\ n f(V(p)) 7 ^?}) = 1. Moreover, since D is correct, for every n E N, 
FD n C Sf n and CD n C Sc n . Thus for every n E N, P(FAmb n U CAmb n ) = 1 — P(Sf n U 
Sc n ) < 1 — P(FD n U CD n ) and lim n — ^oo P(FAmb n U CAmb n ) < 1 — liminf n — >oo P({p € 
SR„ | D(V{p)) 7 ^?}) = 0. This shows that A is FA-diagnosable. 

Assume now that A is FA-diagnosable and has n states. From FA(M) = (S, sq, A, F) 
the FA-automaton of A, we define D = ( S , £ 0 , So, up, Df m ) the finite-memory diagnoser 
where up(s,a) = s' if (s,a,s / ) E A, Df m ((U,V )) = T iff U = 0 and Df m ((U,V )) = A 
iff V = 0. Let us check that D is an FA-diagnoser, and that its size is at most 2 n if n 
denotes the number of states of A. 

commitment When U is empty, it remains empty forever which implies commitment. 

correctness Let w E £* be an observed sequence. If (U, V ) is the state in FA(M) 
reached after reading w, then recall that U (resp. V) is the set of states in A 
that can be reached by correct (resp. faulty) signalling runs labelled by w. By 
construction, if D{w ) = T then w is surely faulty, and if D{w ) = _L then w is 
surely correct. 

reactivity Let p be a signalling run such that D(V(p)) =?. Due to the characterisation 
of Proposition |4.4| the strictly connected component of Mfa that p has reached 
cannot be a BSCC. So given some n E N, 

P({p E F2 | 3m > n D(V(p± m ) =?}) < P({p G Fl \ p^ n does not reach a BSCC}). 
Thus 

P({p e ^ | AnfCP(p)) =?}) = hm P({p E Fl | 3m > n D(V{pi m ) =?}) 

n—>00 

< limsupP({p E F2 | pi n does not reach a BSCC}) 

n—>■ OO 

= 0 . 

size D has at most 2 n memory states because every state of FA(M) consists of a pair 
(U, V) with U C Q c and VC Qj. 

□ 
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As the pLTS of Figure 4.18 is FA-diagnosable, and since any FA-diagnoser is also an 
FF-diagnoser, using Proposition 4.13 we obtain the following lower bound for the size 
of FA-diagnosers. 


Proposition 4.15. There is a family {^4 n }neN °f FA -diagnosable pLTS such that A n 
has 2n + 2 states and it admits no FA-diagnoser with less than 2 n memory states. 


3.3 lA-diagnoser 


We end with lA-diagnosability and build an lA-diagnoser similarly as to what was done 


in Proposition 4.12 


Proposition 4.16. If A is a I A- diagnosable pLTS with n c correct states and nj faulty 
states, one can build an IA -diagnoser with at most 2 Uc 3 n f states. 


Proof. Let A be an lA-diagnosable pLTS. From IA(^4) = (S, sq, A, F) the I A-automaton 
of A, we define D = (S, X 0 , so, up, Df m ) the finite-memory diagnoser where up(s, a) = s' 
if (s, a, s') E A, D fm {{U , V, W)) = T iff U = 0 and D fm ({U, V, W)) = J. iff W = 0. Let 
us prove that D is indeed an lA-diagnoser for A. 


commitment. When U is empty, it remains empty forever which implies commitment. 

correctness. For any word w E £*, we denote by (U w ,V w ,W w ) the state in IA(M) 
reached after reading w. For any word w, if U w = 0, by construction of IA(M), 
w is surely faulty. Assume now that W w = 0 and U w A 0- Let w' be the longest 
proper prefix of w such that W w > = 0. Let p be any signalling run with V(p) = w. 
Assume that pu w n is faulty. Thus the states visited by p^ n for |u/| < n < |rc| were 
always in W Plri . Since W w = 0, this is not possible and so p±\ w t\ is correct. Thus 
every time a state with W = 0 is visited, the length of the greatest prefix, for 
which all signalling subruns corresponding to this prefix are correct, is increased. 
This establishes correctness. 


reactivity. Let p be an infinite run such that D sup (V(p)) =?. Due to the characterisa¬ 
tion of Proposition 4.5 either (1) the strongly connected component of A\/\ that 
p infinitely often visits is not a BSCC or (2) p does not visit infinitely often all 
the states of this strongly connected component. The probability of such runs is 
null which establishes the reactivity. 


size D has at most 2 nc 3 n f memory states because every state of IA(^l) consists of a 
triple (U, V, W) with U C Q c and V U W C Qf. Moreover, one does not keep in 
V the states that are tracked in W, ensuring V FI W = 0. □ 


The following lower bound can be derived from the proof of Proposition 4.13 


since 


the pLTS of Figure 4.18 is lA-diagnosable, 
FF-diagnoser. 


and because any lA-diagnoser is also an 
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Proposition 4.17. There is a family {-4 n }neN °f IA -diagnosable pLTS such that A n 
has 2 n + 2 states and it admits no IA -diagnoser with less than 2 n memory states. 

The construction of an exact diagnoser can thus require exponential time, which is 
one class above the verification of exact diagnosability. This exponential time is only 
necessary if we want to build the full diagnoser though. Another possibility would be 
to update the state of the diagnoser on-the-fly during a run. One would only need to 
keep the current memory state (which has linear space), but the update process would 
take a polynomial time, instead of the constant time obtained when using a fully-built 
diagnoser. 

4 Conclusion 

In Section [lj we gave characterisations of the different notions of exact diagnosability 
and of one notion of approximate diagnosability for finite systems. The characterisations 
of the notions of exact diagnosability are of the same descriptive complexity, something 
that is impossible in the general case as shown in Section [3] of Chapter i page 
The characterisation of the notion of approximate diagnosability has two important 
differences compared to the ones of exact diagnosability: ( 1 ) it depends on the exact 
probabilities of the system and ( 2 ) it only requires a comparison between pairs of states. 

In Section [2j we gave matching upper and lower bounds for the various diagnos¬ 
ability problems for finite systems. These results heavily relied on the characterisations 
given in Section [lj Thus, as the characterisations of every notion of exact diagnosabil¬ 
ity have the same descriptive complexity, it is not surprising that they end up having 
the same complexity. The results on the approximate diagnosability notions are more 
surprising. AFF-diagnosability, the notion for which a characterisation was obtained, is 
decidable in polynomial time, a better complexity than what is needed for the exact 
diagnosability notions. This gain in complexity is obtained as, contrary to exact diag¬ 
nosability where one needs to follow sets of states, for AFF-diagnosability, only pairs of 
states have to be compared. The comparison however depends on the exact values of 
the probabilities of the system, which brings a different kind of difficulty to the analysis. 
While this difficulty could be solved for AFF-diagnosability, it is the main reason of the 
undecidability of all the other notions of approximate diagnosability: (uniform) eFF- 
and uniform AFF-diagnosability. On this point, uniform AFF-diagnosability is equiva¬ 
lent to the notion of AA-diagnosability introduced in [TT05| which decidability was left 
open (only necessary conditions were given). Our undecidability result thus answers 
negatively to this question. 

In Section [3j we gave automatic methods to construct finite-memory diagnosers for 
systems that are exactly diagnosable. We also showed that the sizes of the generated 
diagnosers are asymptotically optimal. For approximate notions of diagnosability, there 
does not necessarily exist finite-memory diagnosers and deciding the existence of such 
a diagnoser is undecidable. When such a diagnoser does not exist, one may need un¬ 
bounded memory. Depending on the form of the unbounded memory required, it can 
be more or less manageable (for example a counter can easily be implemented). 
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Chapter 5 

Algorithmic analysis of the 
diagnosability of infinite pLTS 


In Chapter [tj we showed how to solve diagnosability for systems that can be represented 
by a pLTS with finitely many states.While this encompasses many kinds of systems, 
this is far from being exhaustive. Often, in order to satisfy its specification, a system 
will require unbounded memory: for example, when the system receives and records 
information or requests from the environment. Observe that an infinite number of 
states does not mean an infinite memory per se, but only an unbounded one. Stacks 
and queues are instances of such dynamic data structures. 

While allowing an infinite amount of states increases the expressive power of the 
model, it increases the difficulty of its study. First, the studied systems must possess 
a finite representation. This can be done by assuming only a finite part of the system 
needs to be studied ( e.g. infinite systems with finite attractors |BBS06j ) or by using 
a higher level model whose semantics is an infinite-state system (Petri nets |Mur891 
IDia091 |CGS14| . well structured transition systems |FS0l| . pushdown automata |AM041 
IKEM061IMP09IIHS101IEY12] ). We study such a formalism in this chapter. 

Diagnosability has already been studied for infinite-state non-stochastic systems: 
represented by pushdown automata [MP09J or by Petri nets |CGLS12l IBHSS18] . Par¬ 
tially observable visibly pushdown automata is a subclass of partially observable push¬ 
down automata for which diagnosability was studied in fMP09| : for such models, di¬ 
agnosability is decidable (using the determinisation procedure of |AM04| ). With a 
restriction on the unobservable subnet akin to our convergence assumption, |COLS12] 
gives a decidable characterisation of a non-stochastic notion of diagnosability for par¬ 
tially observable Petri nets. However the algorithm is non-primitive recursive. The 
authors of |BHSS18j extend this work by considering different classes of Petri nets and 
reducing the complexity (EXPSPACE for the general case). However to the best of our 
knowledge diagnosis of probabilistic infinite-state systems has not yet been studied. 

So we extend these works by considering the probabilistic variants of diagnosabil¬ 
ity. In Section [TJ we study the stochastic diagnosability of partially observable prob¬ 
abilistic pushdown automata (POpPDA). As diagnosability is already undecidable for 
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non-stochastic systems |MP09j . the decidability of the stochastic variants is unlikely. 
However the notations introduced here will be used in Section [2] where we consider a 
restriction of POpPDA called partially observable probabilistic visibly pushdown au¬ 
tomata (POpVPA). This restricted class has many advantages. It naturally benefits 
from the numerous results which are known for POpPDA |BEKK13j . especially the 
ones on model-checking algorithms [KEM06] !EY12j . Moreover, the authors of |AM04| 
gave an algorithm for the determinisation of a POpVPA. PopVPA generates an infinite- 
state pLTS and the efficient characterisations given in Chapter [4] strongly rely on the 
finiteness of the models. They thus cannot be used any more. So, we use the char¬ 
acterisations from the Section [3] of Chapter [3] based on the pathL logic. However the 
model-checking algorithm cannot be directly applied to the formulae of the pathL logic. 
Some tricky machinery will be required to “encode” the path formulae of the pathL logic 
in the system in order to use the results of [ EY12] , Finally, in Section |3j we study 
the diagnosability of partially observable stochastic Petri nets (POSPN), we mimic the 
restriction used on POpPDA in Section [2j and discuss the case of partially observable 
stochastic visible Petri nets (VSPN). 

This chapter develops and extends some of the results from |BHL16bl iLGSIBj . 


1 Diagnosability of probabilistic pushdown automata 

In this section we study infinite-state pLTS generated by probabilistic pushdown au¬ 
tomata (pPDA). First, we define pPDA and the infinite-state pLTS generated by a 
pPDA. The pPDA model being very expressive, we show that diagnosability of pPDA 
is undecidable. 


1.1 Probabilistic pushdown automata 

A pPDA randomly generates infinite behaviours using a stack. This stack contains 
letters of a stack alphabet with the most recently added letter put at the top. Transitions 
of the pPDA can be conditioned by the top of the stack. Moreover, a transition can 
push a new element onto the stack, pop one element off it or modify the top of the 
stack. Let us first see an example of an infinite state pLTS, that we will be able to 
represent by a pPDA. 


Example 5.1. The pLTS of Figure 5.1 represents a server that accepts jobs (event in,) 
until it randomly decides to serve the jobs (event serve). When a job is done the result 
is delivered (event out,). When all jobs are done, the server waits for a new batch of 
jobs. However randomly, the server may trigger a fault (event f) and then abort all 
remaining jobs (event abort,). Afterwards, the server is reset (event reset,). 

The infinite number of states in this system comes from the unbounded number of 
jobs the server can receive before he starts serving them. To see this system using a 
stack, one could use a stack letter to represent a job and add one such letter to the stack 
every time a new job comes in. Dealing with a job (event out,) or aborting it (event 
abort,) removes an element of the stack. 
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reset 



Figure 5.1: An infinite-state pLTS that can be represented by a pPDA. 


We now define pPDA similarly to what can be found in |KEM06j . 

Definition 5.1. A probabilistic pushdown automaton (pPDA) is defined by a tuple 
V = (Q, qo, £, T, 6, P) where: 

• Q is a finite set of states with qo the initial state; 

• £ is a finite alphabet of events; 

• r is a finite alphabet of stack symbols including a set of bottom stack symbols 
with initial symbol _l_o £ r_i_; 

• 5 C QxrxExQxr* is the set of transitions such that for every ( q , 7 , a, q, w ) E 5, 
M < 2, 7 E Tj_ implies w E r_i_(r \ r±)* and 7 ^ T_l implies w E (r \ Tj.)*,' 

• P is the transition probability function fulfilling for every q E Q and 7 E T: 

Y ' p [(q,'y,a,d,w)] = i- 

{q,l,a,q',w)£5 

A pPDA may be viewed as a pLTS equipped with a stack. The transitions of 
the pPDA can depend on the top symbol of the stack and modify it. The definition 
ensures that the stack is never empty: the bottom stack symbols Tj_ are never removed. 
Moreover symbols of Pj_ never occurs elsewhere in the stack. Let T = (q, 7 , a, q ’, w) E S 
be a transition of a pPDA. If |rc| = 1 (resp. \w\ = 2, |m| = 0) then T is said to be a local 
(resp. push, pop ) transition. A local transition can update the top symbol and a push 
transition can modify the top symbol and add another symbol on top of it. Notions 
such as runs are defined on pPDA analogously to what was done for pLTS. We call 
configuration the pair composed by a state and a stack. 

The semantics of a pPDA is the (potentially infinite) pLTS representing its be¬ 
haviour. The states of this pLTS are pairs consisting of a state and a stack contents. 
They therefore contains all the information that are necessary in the pPDA to determine 
the available transitions and their probabilities. 
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Definition 5.2. A probabilistic pushdown automaton pPDA V = (Q, go, S, T, <5, P) de¬ 
fines a pLTS Av = ( Qv , (qo, -L), £, Ty, Py) where: 

• Qv = {{q,z) \ q £ Q Az £ J_r*}; 

• Tv = {((<?, 27), a, (g', zw )) | 2:7 G -LT* A (g, 7, a, g', w) G 5 }; 

• For every ((q,zy),a, {q',zw)) G T V) P v [((g, 27), a, (g',:zw))] = P[(g, 7, a, g', w)]. 


As a pPDA has a finite number of states, the associated pLTS is finitely branching. 


Example 5.2. Figure [A"2] gives an example of a pPDA whose semantics is the pLTS 


from Figure 5.1. Indeed, the stack alphabet has only one letter. We could thus replace 


it by a counter giving the number of element in the stack. The pLTS of Figure 5.1 does 
exactly that by representing the configuration (gi,7 n ) of the semantics of the pPDA by 
the state q\ n and similarly for the other configurations. A transition t = (g, 7 , a, q',w) 
is represented by an edge from state q to state q’ and labelled by P[t] • 7 ,a,w. 

Observe that in this example the set of states is not partitioned between faulty and 
correct states as from the state f\ reached by a faulty run, one can go back to the initial 
state with the reset event (event r). 



in 


(go, | J-o |) —^ (go, 
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Figure 5.2: A pPDA generating the pLTS from Figure 5.1 and two finite of its runs. 


As for pLTS, we enlarge pPDA with partial observation features. As discussed in 
Section 1.3 of Chapter [2j this can be done either by partitioning the alphabet of events 
into observable and unobservable events or by providing a mask function associating 
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with every event an observation. In the previous chapters we used the partition of the 
alphabet of events. Here, we use the mask function which is more appropriate for our 
needs. 

Definition 5.3. A partially observable pPDA (POpPDA) is a tuple (V, Tj 0 ,P) consist¬ 
ing of a pPDA V equipped with a mapping V : £ —> £ 0 U {e} where E 0 is the set of 
observations. 

A POpPDA is diagnosable according to a notion of diagnosability if the pLTS it 
generates is diagnosable. As the generated pLTS are finitely branching, thanks to 
Theorem |3.1| a POpPDA is FF-diagnosable iff it is IF-diagnosable. 

Example 5.3. Consider the pPDA V of Figure \5.2\ we define E 0 = {in, out, loc, reset} 
and two observation masks V\ and V -2 with Pi (in) = in, Pi (serve) = Pi (empty) = 
Pi (reset) = loc, Pi (abort) = Pi (out) = out and Pi(f) = £, P2(reset) = reset and for 
every event t / reset, P2 (t) = Pi (t). (V, E 0 ,P 1) and (V, E 0 ,P2) are two POpPDA 
differentiated only by the observation of the event reset. As a faulty run will inevitably 
contain a reset and a correct run that leaves qo will contain a serve, the POpPDA 
(V,S 0 ,P 2 ) which distinguishes these two events is diagnosable for every non-uniform 
exact notion of diagnosability. However, the POpPDA (V, S c , Pi) is not diagnosable as 
serving the requests and going back to qo has the same observation as making a fault, 
aborting the requests and going back to qo. 


1.2 Undecidability of diagnosability for POpPDA 


Unfortunately, for every notion of diagnosability, the diagnosability problem for POp¬ 
PDA is undecidable. The undecidability can be obtained by adapting the proof for 
diagnosability of non-probabilistic pushdown automata |MP09| . However, in order to 
show how robust the result is, we rather reduce from the Post Correspondence Prob¬ 
lem (PCP). An instance of PCP is given by an integer n G N and two families of 
non-empty words {u*}j< n and on the alphabet {a, b}. The following question 

is undecidable (Pos46j : does there exist k > 0 and i\,...ik & {l,...,n} such that 


, w, 


' l/ k 


.Vi 


1k 


We show in Theorems 5.1 and 5.2 that undecidability already holds for two 


in¬ 


comparable) subclasses of POpPDA with restriction on what is observable and on the 
number of phases of any run. A phase is a portion of a run in which the stack either 
never decreases or never increases. 


Theorem 5.1. The diagnosability problems are undecidable for POpPDA even when 
(1) a local transition does not update the top of the stack, (2) every event labelling a 
piLsh transition is fully observable and corresponds to the pushed symbol, and (3) every 
run has at most two phases. 

The use of the stack is obviously central to the proof of the undecidability. Moreover 
conditions ( 1 ) and ( 2 ) limit the ability of push and local transitions to silently manip¬ 
ulate the stack. There is no such limitation for pop transitions however as the proof of 
undecidability heavily rely on hiding when the pop transitions are performed. 
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Let us sketch the proof. We reduce the PCP problem to the diagnosability problem. 

To do so, the POpPDA we build, from a PCP instance (n, {vi}i< n , {wi}i< n ), is divided 
in three parts. In the first one, it will select and push onto its stack a sequence of 
numbers, i\ ... ifc, with Vj < k, ij < n. Then it goes randomly to one of the two other 
parts of the POpPDA, one part being accessed by a faulty transition. Each of these two 
parts is associated with a family of words of the PCP instance. Once it reached one of 
these parts, say the one associated with the Wi, the run will read the words of Wi induced 
by the numbers pushed on the stack. The resulting observation is w = tcq ... Wi k . On 
the other part, the observation is similarly v = ... Vi k . The fault can be detected if 

and only if v 7 ^ w. Having the pop transition undetected is fundamental as it allows to 
hide when the word Wi j starts being read. 

Proof. Let (n, {vi}i< n , {wi}i< n ) be a PCP instance In this proof, we let C (resp. mf) 
be the length of Vi (resp. wf). Also, given a word w and k < |iy| we use w[k] to denote 
the k th -letter of w. 

We build a pPDA V = ( Q , £, T, 5 , P) as follows: 

• Q = {<?o, <?c q s , fs} U | 1 < i < n, 1 < k < fi} U {/f | 1 < i < n, 1 < k < ; 

• S = { 1 ,.. .,n,\\,u,r,f,a,b}] 

• r = {l,...,n, T 0 } with T_l = {T 0 }; 

• 6 consists of the following transitions: 

{(q 0l ± 0 ,x,± 0 x,q c ) | 1 < x < n} U {(q c , x, y, xy, q c ) | 1 <x,y<n} 

U {(qi,z,Vi[k],z,q!? +1 ) \ 1 < * < n, 1 < k < £ h z G (T 0 ,1,... ,n}} 

U {(fi,z,Wi[k],z,ff +1 ) \ 1 <i <n,l < k < rrii,z G (T 0 ,1,... ,n}} 

U {(qi,z,Vi[£i],z,q s ) \ 1 < i < n, z G (T 0 ,1,... ,n}} U {(q s ,x,r,e,ql) | 1 < x < n} 
u {{fr^ z ^ w A m i i z ,fs) | 1 < i < n,z G {T 0 ,1,... ,n}} U {{f s , x, r, e, f£) | 1 <x<n 
U {(q c ,x,u,x,q s ),(q c ,x,f,x,f s ) \ 1 < x < n} U {(q- s , T 0 , t], T 0 , q- s ), (/ s , T 0 , t|, T 0 , / s )>- 

• P assigns arbitrary positive probabilities to transitions in 5: 

P(?, 7) a , q' 1 w) > b yy (q, 7 , a, q , w) € 5 and ^ P[(q, 7 , a, q', w)] = 1. 

(5,7,a,q',io)S <5 

We further consider the POpPDA (V,S n ,P) with E 0 = E \ {r, u, f}, and the masking 
function satisfies V{u) =V{r) = V(f) = s and V{x) = x for any other event x. This 
POpPDA is represented in Figure [5~3| 

Let us prove that the instance of the PCP is positive if and only if the POpPDA is 
IF-, IA-, FA- and AFF-diagnosable. 

First, observe that t] almost surely occurs in an infinite run of the pPDA V. Thus, for 
any e > 0 , there exists iV £ £N such that the measure of signalling runs with observable 
length N e that reach configurations (q s , To) or (/ s , To) by an event \ is at least 1 — e. 
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• Assume that there exists a solution i%,..., to the PCP instance ( n , {wi}i< n ). 

Consider in the POpPDA the faulty run: 

Pf = Qo(ijqc)j<kf(fsr(ff.w ij \p]) p < rni .) j <k{fs\\) u , 
and the correct run: 

Pc = qo(ijq c )j<ku(qsr{qi j Vi j \p])p<e i .) j <k{qs\\) ul ■ 



Figure 5.3: A POpPDA for the proof of Theorem 5.1 


These two runs have the same observed sequence: V(pf) = V(p c ) = i\... ik'w'tt 1 with 
w = Wi l ... Wi k = v t] ... Vi k . Therefore, pf is an infinite ambiguous faulty run. Given 
that P (pf) > 0, we deduce that the POpPDA (V,S 0 ,P) is not IF-diagnosable. From 
Theorem |3.1| it is also neither lA-diagnosable nor FA-diagnosable. Moreover, after the 
occurrence of a fault, there is no probabilistic choice. As a consequence the correctness 
proportion is either 0 or 1/2. As the correctness proportion of the faulty prefixes of pf 
is never 0 as seen above, the POpPDA is not AFF-diagnosable. 
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• Conversely, assume that the PCP instance (n, {vi}i< n , {wi}i< n ) has no solution. Let 
e > 0, let N e € N be the integer obtained with our earlier observation. Consider a 
correct run p c with observable length N e . ending in (q s , _l_o) and containing at least an 
occurrence of t|. Its observed sequence is of the form V(p c ) = i\ ■ ■ - ik v i\ ■ ■ - v i k for 
some i\,..., ik , m. Due to the fact that (n, {vi}i< n , { Wi}i < n ) has no solution, no faulty 
run can have the same observed sequence. Therefore, p c is surely correct. Symmetrically, 
any faulty run ending in (/ s ,_l_o) after an occurrence of \ is surely faulty. We thus 
conclude that, for any e > 0, there exists N e E N such that P(FAmb 7 v £ C CAmb 7 v £ ) < e. 


As a consequence, the POpPDA (V, Y, 0 ,V) is FA-diagnosable. By Theorem 3.1 it is also 
lA-diagnosable, IF-diagnosable and AFF-diagnosable. D 

A similar undecidability result holds for a class of POpPDA in which pop events are 
fully observable, and the number of phases is constant: 


Theorem 5.2. The diagnosability problems are undecidable for POpPDA even when 
(1) a local transition does not update the top of the stack, (2) every event labelling a 
pop transition is fully observable and corresponds to the popped symbol, and (3) every 
run has at most two phases. 


Proof. The proof follows the same line as the one for Theorem 5.1 The difference is 


that instead of choosing first the words that will be read by pushing them on the stack 
and later popping them off discreetly, the pPDA reads the words and silently push on 
the stack which words were read and at the end pop them off and verify if the same 
sequence could indeed be used for both family of words. 

From an instance (to, {uj}j< n , {wi}i< n ) of PCP, let us define a pPDA V = (Q, S, T, S, P) 
where: 


• Q = Uo, Qs: fs, q e , fe}0{qf \ l<i<n,l<k< A}u{/f \ 1 < i < n,l < k < mj; 

• £ = {1,.. .,n,\\,u,c,f,a,b}-, 

• r = {1,. . ■ ,n, J_o} with r ± = {_l 0 }; 

• 5 consists of the following transitions: 


{(90, -L, u, T, q s ), ( q 0 , -L, f, T, / s ), (q e , _L, t|, T, q e ), (/ e , _L, t], T, f e )} 

U {(qi,z,Vi[k],z,q!f +1 ) \ 1 < i < n, 1 < k < z G {i, 1,..., n}} 
u {(fi,z,Wi[k],z,f^ +1 ) | 1 < i < n, 1 < k < m u z G {_L,1,. ..,n}} 

U {{ql\z,Vi[ii],z,q 3 ) | 1 < * < n,z G {_L, 1,... ,n}} 
u {(fr^ z ’ w i[ m i\’ z ^fs) | 1 < * < n,z G {T,l,... ,n} 

U {( q s ,z,c,zx,ql ) | z G {l,l,...,n},iG {!,-• • ,n}} 

U {( f s ,z,c,zx,fx ) | z G {T, 1,... ,n},x G {1,... ,n}} 

U {(q s ,x,x,£,q e ) | x G {l,...,n}}U {(/ s , x, x, e, f e ) \ x E {1,..., n}} 
U {(q e ,x,x,£,q e ) | x G {1,... ,n}} U {(/ e , x, x, e, f e ) \ x G {1,..., n}}. 
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• P assigns arbitrary positive probabilities to transitions in <5. 

We further consider the POpPDA (V,T, 0 ,V) with £ 0 = X \ {c, u,f}, and the masking 
function satisfies V(u) = V(c) = V(f) = £ and V(x) = x for any other event x. This 
POpPDA is represented in Figure [A4| 



Figure 5.4: A POpPDA for the proof of Theorem 5.2 


Let us prove that the instance of the POP is positive if and only if the POpPDA is 
IF-, IA-, FA- and AFF-diagnosable. 

First, observe that t| almost surely occurs in an infinite run of the pPDA V. Thus, for 
any e > 0, there exists N e 6 N such that the measure of signalling runs with observable 
length N s that reach configurations (g e ,_l_o) or (/ e >-Lo) by an event t] is at least 1 — e. 
• Assume that there exists a solution to the PCP instance (n, {uj}j< n , {rcj},< n ). 

Consider the faulty run: 

and the correct run: 

p c = qouqs(c{qi.Vi j \p\) p <e i .q s ) j < k (i j q e ) j < k (\\q e ) UJ . 
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These two runs have the same observed sequence: V(pf ) = V(p c ) = wi\ .. .if .with 
w = W{ , .. .Wi k = v tl ... Vi k . Therefore, pf is an infinite ambiguous faulty run. Given 
that P (pf) > 0, we deduce that the POpPDA (A,Tj 0 ,V) is not IF-diagnosable. From 
Theorem |3.1[ it is also neither lA-diagnosable nor FA-diagnosable. Moreover, after 
reaching the state f e or q e , there is no probabilistic choice. As a consequence the 
sequence of the correctness proportion of the faulty prefixes of pf is stationary. As it is 
never 0 as seen above, the POpPDA is not AFF-diagnosable. 

• Conversely, assume that the PCP instance (n, {wi}i< n ) has no solution. Let 

e > 0, let N e € N be the integer obtained with our earlier observation. Consider a 
correct run p c with observable length N e ending in (g e ,_l_o) and with an occurrence 
of tj. Its observed sequence is of the form uq ... Vi k i\ ... ik\\ m for some i \,..., m. 
Due to the fact that (n, {vi}i< n , {wi}i< n ) has no solution, no faulty run can have the 
same observed sequence. Therefore, p c is surely correct. Symmetrically, any faulty run 
ending in (/ e , To) by an occurrence of t is surely faulty. We thus conclude that, for any 
e > 0, there exists N e £ N such that P(FAmbqr £ tt) CAmbqr e ) < e. As a consequence, 
the POpPDA (V,T, 0 ,V) is FA-diagnosable. By Theorem |3.1| it is also lA-diagnosable, 
IF-diagnosable and AFF-diagnosable. □ 

2 Diagnosability of probabilistic visibly pushdown automata 

As diagnosability is undecidable for pPDA, we now turn to a more restrictive model: 
probabilistic visibly pushdown automata (pVPA) |AM04j . While keeping a signifi¬ 
cant expressive power, pVPA is a natural subclass of pushdown automata that is more 
tractable than the general model and which language has many of the desirable prop¬ 
erties that regular languages have. In particular, there exists a method for the deter- 
minisation of a non-deterministic visibly pushdown automaton |AM04| . 

We formally define pVPA and describe how to build a diagnosis-oriented determin- 
isation of a pVPA. Then, we give a decision procedure for diagnosability and study the 
hardness of the diagnosability problems. 

2.1 Probabilistic visibly pushdown automata and diagnosis-oriented 
determinisation 

A pVPA is a pPDA where events are partitioned into three sets depending on if they 
correspond to push, pop, or local transitions. 

Definition 5.4. A probabilistic visibly pushdown automaton (pVPA) is a pPDA V = 
(Q, qo, £, T, 6, P) whose event alphabet is partitioned into local, push and pop events 
£ = l±l Ejj tt) E[, and such that for every transition T = (q , 7 , a, q', w ) & 5, T is a local 
(resp. push, pop) transition iff a is a local (resp. push, pop) event. 

A p VPA without the transition probability function P is called a visibly pushdown 
automata (VPA). 

The definitions of pPDA carry on to pVPA in particular the semantics of a pVPA 
is an infinite-state finitely-branching pLTS. 
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Example 5.4. Consider the pPDA of Figure \5.2\ This pPDA is apVPA as shown by the 
partition of events given by Ej = {in}, E b = {out, abort} and = {serve, empty, reset , f}. 

To define partially observable pVPA, we equip a pVPA with a mask function and 
require that only local events may be unobservable, and that pushes and pops can still 
be distinguished. As a consequence, given the observed sequence of one run, one can 
deduce the height of the stack as the difference between pushes and pops, plus one (the 
bottom symbol). 

Definition 5.5. A partially observable pVPA (POpVPA) is a tuple (V,H 0 ,V) consist¬ 
ing of a pVPA V equipped with a mapping V : E —> E 0 U {e} such that: 


• E 0 = E 0jb l±J E 0j (j l±J E 0 b is the set of observations; 


• V(Yif) C E 0it] U {e}, V(T, (i) C E 0 j and P(E b ) C E 0]b . 

When we aimed to verify the diagnosability of finite pLTS in Chapter [4] one of the 
first step was to build a diagnosis-oriented determinisation of the pLTS (see Defini¬ 


tion 4.1 page 91). While this could not be done for pPDA, a determinisation for pVPA 


was established by Alur and Madhusudan |AM04| . Following the same approach, we 
now explain how to adapt the determinisation of I AM 041 for diagnosability. For a pVPA 
V, its determinisation is called the estimate VPA of V and is denoted M(V). As in the 
finite case, we need tags that reflect the category of runs (faulty or correct) given an 
observed sequence with a distinction between “old” and “young” faulty runs. Due to its 
technicality, we postpone the formal definition of M(V): we first explain some features 
of the construction and illustrate them on an example (represented in Figure [575] ). 
States and stack symbols. The VPA A(V) tracks all runs with same observation in 
parallel memorising their status w.r.t. faults. More precisely to the current set of runs 
corresponds the symbol on the top of the stack which is a set of tuples where each tuple 
is written as a fraction . Let us describe the meaning of this tuple: 

• q is the current state of the run and 7 is the symbol on the top of its stack; 

• X S Tg = {U,V,W} is the status of the run: U for a correct run, V for a young 
faulty run and W for an old faulty run; 


• The denominator ( 7 ~,X~, q~), is related to the configuration just after the last 
push event of the run: 7 “ is the stack symbol under the top symbol, while X~ is the 
status of the run reaching this configuration and q~ the state of this configuration. 


A priori, a single state run would be enough. However the simulation of a pop event 
in the original VPA is performed in two steps requiring some additional states that we 
explain later. 


Example 5.5. The initial configuration of the VPA M(V) of Figure 5.5 (run, {} ) 
corresponds to the empty run represented by a singleton. The denominator of bottom 
stack symbols is by convention (_l_o> U,go) an d is irrelevant for specifying the transitions 
of A(V). 
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n x 

a o 


r -L,X,<?q 
1 -l,X,g 0 




r 7,X,go 
1 -LjX,go 


r 7.x,go 
’-7.x, go 




r 7,X,gi 
1 -L,X,g 0 




r 7,X,gi i 
1 7,X,g 0 J 


X _ f -L,X,gi ±,X,/i -j X _ r 7,X,gi 7,X,/i 1 X _ r 7,X,gi 7,X,/i 1 
1 _L,U,go ’ -L,u,g 0 J ’ 1 “l _L,X,g 0 ’ X,X,g 0 J ’ c °o — 1 7,U,go ’ 7,U,go J > 


X e {U, W} 








r7,u,goi 

1-7, U,g 0 -* 

r 7,U,go i 

1 _L 0 ,U,go J 


r 7,U,gi i 

1-7,11, g 0 J 

r 7 ,U,go l 

1 -to,U,go J 

{run, 

r -l-o,U,g 0 1 
1 -L 0 ,U,go J 

. in , 

J — ► {run, 

r 7,U,go i 
f -Lo,U,go J 
r -Lo,U,go i 

1 _Lo,U,g 0 J 

\ hi , 

) — {run, 

1 loc t 

) — {run, 

r -Lo,U,go i 
1 -l-o, U,go J 

r J_o,U,goi 

1 -l-o,U,go J 


pop 


({ 


U,gi W,/i 


-Lo,U,go ’ J-o,U,go 




r ,u,gi W i 
l-Lo,U,go’ -Lo,U,go-* 
r -Lo,u,» i 

1 Jtn.U.On J 


loc 


run, 


~ -l-o, U,go - 

r -Lo,U,go 
l-L 0 ,U,g 0 


)-({ 


U,gi W,/i -I 

7,U,go ’ 7-U,go ’ 


}) 


( run /Ma -Lq.w./i -i \ 

P aa ' l-Lo,U,qo ’ - 1 - 0 ,U,go J 1 


reset 


(™ n > ) 


r ~;,U-gn 1 
1 -l-o,U,go J 
r -Lo,U,go i 
1 .L 0 ,U.go ‘ 


Figure 5.5: The VPA A(V) associated with the POpVPA (V,Yi 0 ,V 2 ) of Example 5.3 
with two runs. The tag V was ignored to remove redundancy and simplify the figure. 
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Tag updates. Let us explain how the tag X of an item of the current stack 

symbol is determined. If this item corresponds to a correct run then X = U. When after 
a transition of A(V) a (tracked) correct run becomes faulty, there are two cases. Either 
there was no tag W in (the numerators of items of) the top stack symbol of the stack 
then the run is tagged by W. Otherwise it is tagged by V meaning that it is a young 
faulty run. The tag V (young) becomes W (old) when, in the previous state, there was 
no tag W in the top stack symbol. A tag W is unchanged along the run. 

Local transitions. Given an observed local event o £ £ 0 ) t|, from the state run with top 
stack symbol bel, there is a local transition ( run,bel,o,run,bel') in A(V) looping over 
run that encodes the possible signalling runs with observation o in V. More precisely 
for every transition sequence ( q,a ) (r, /3) in V (he. a sequence of unobservable local 

events ended by an event e with Vie) = o) and A22i q ^ fre/ one inserts -JrfrA _ in 
bel'. The value of Y follows the rules of tag updates. 


Example 5.6. In the VPA A(V) of Figure 5.5 there are several transitions correspond¬ 
ing to the transition (qo, 7 , serve, qi, 7 ) of V including (run, }, loc, run, })• 

For example, the runs represented in Figure |5.5| use this transition. 


Push transitions. Given an observed push event o £ E 0 j, from the state run with top 
stack symbol bel, there is a push transition (run, bel, o, run, bel'bel") in A(V) looping 
over run that encodes the possible signalling runs with observation o in V. More precisely 
for every transition sequence (q, a) A- (r, j3~ (5) in V and Q _ Q! ^ g )? _ ^ bel one inserts 

g- x—g— bel' and r hi bel". The value of Y follows the rules of tag updates. 


Example 5.7. In Figure 5.5 several transitions of A(V) correspond to the transition 
(qo, -L 0 , in, g 0 , -L 07 ) ofV, including (run, {in, run, { }{}) and sev¬ 

eral transitions of A(V) correspond to the transition (qo, 7 , in, qo, 77 ) ofV, including 
(run, { j? 0 U u'^ 0 1 , in, run, { -g’ U ^° o }{ ffjrt i) • Here > the specification of the tag updates is 


- 7 >U, 9 o- 

straightforward since it does not involve faulty runs, 
use these two transitions from the initial state. 


The runs represented in Figure 5.5 


Pop transitions. Given an observed pop event o £ \,, from the state run with top 

stack symbol bel, the “pop operation” is performed by a sequence of two transitions: 
a pop transition labelled by o reaching another state that contains some information. 
This information is then used by the next (local) transition labelled by e to move back 
to state run with a consistent stack symbol. Given an intermediate stack symbol, there 
is exactly one possible such transition. Thus despite these transitions, A(V) is still 
deterministic. The first transition ( run,bel,o,£,e ) in M(V) is specified as follows. The 
next state f is a set of items of the following shape Q _ . More precisely for every 

transition sequence (q, a) A- (r, e) in V (i.e. a sequence of unobservable local events 
ended by an event e with V(e) = o ) and ffffl £ one inserts _ —— in i. The 

value of Y follows the rules of tag updates. A transition (l, bel, e, run, bel') is specified 
as follows. For every in I and _ in bel (i.e. the denominator of the first 

fraction and the numerator of the second fraction match), one inserts 2 x-^- i n bel'. 
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Example 5.8. Let us describe how the pop is performed by two transitions in the runs 
of the VPA of Figure \ 5. 5| from the state reached after event serve. From q\ with 7 as 
top of the stack there are two transitions whose observation is pop: (q 1 , 7 , out, < 71 , e) and 
(gi, 7 , abort, / 1 , e). Thus starting from run with top stack symbol }, one reaches 

state £ = { U ^ () , } ■ The faulty run is tagged with W as there was no tag W in the 

former top stack symbol. In the next configuration, the top stack symbol is { ^ U j g g 0 }~ 

So the transition labelled by e moves back to state run with updated top stack symbol 
r 7 .U .<?1 TjWJi 1 
f -Lo,U,go ’ J-o-U,go ' 

We now give the definition of the estimate VPA A(V) associated with a given 
POpVPA V. Let fi E {g,c,f} we write (q, 7 ) ( q',w) with o E E 0 if when p = g 

(resp. c, f), there exists a (resp. correct, faulty) run of transitions starting from (q, 7 ) 
to ( q',w ) such that all transitions are unobservable except the last one labelled by e 
with V(e) = o. Let p be such a run then we also write (q, 7 ) =^7^ ( q',w ) All transitions 
of such runs are local except the last one whose type depends on the type of o. 

Definition 5.6. Given (V,V,T, 0 ) a POpVPA with V = (Q, E, T, 6, P), its estimate 
VPA is the deterministic VPA A(V) = (Q e ,13 0 ,T e ,d e ) defined by: 

• Q e = {run} l±J (2 rx ( Tgx ®^ 2 \ 0) is the set of states with initial state qf) = run; 

• r e = 2( rxTgx< ^ 2 \ 0 is the stack alphabet with set of bottom stack symbols Tj_ = 

2 Init \ 0 where I nit = { 7777 ^ | (X, q) E Tg x Q} and initial stack symbol If = 

r goV-Lp i . 
l 9 o,U,± 0 J’ 

• The transition relation 5 e is defined as follows. 
local transitions (run, bel, o, run, bel') E 5 e if: 

• ^ bel 1 iff there exists E bel and (q,a ) =7 C (r,/3). 

• If W occurs in bel, J^f r _ E bel' iff there exists a °L^’ q _ ^ ^ an ^ ^ a ^ 

(r,P). 

• If W occurs in bel, a -'^ r q ~ £ bel' iff 

(1) there exists E bel and ( q,a ) (r,/3) or 

(2) there exists a -’x’ q - ^ bel and (q,a) (r,(5). 

• If W does not occur in bel, E bel' iff 

(1) there exists Q a \' q ~ £ bel and ( q,a ) =7/ (r,/?) or 

(2) there exists a a '^f q q ~ £ bel and ( q,a ) =7 9 (r-, /5). 

push transitions (run, bel, o, run, bel'bel”) E S e if: 

• E 6 eZ 7 and G i/f there exists E bel and (q, a ) =7 C 

(r,/3-fi). 
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• 7/W occurs in bel, '^ q f G bel' and, G bel" iff there exists a a fi[' q - £ 

7eZ and (q,a) =7 9 ( r,/3~/3). 

• If W occurs in bel, G 6e/ / and nl\f G bel" iff 

J ’ a ,X,g p ,\/,r JJ 

(1) there exists G 6el and (q,a) =7/ (r,(3~(3) or 

(2) there exists G 7eZ and (q,a) =7 9 (r,/3~(3). 

• If W does not occur in bel, G 7e^ and n^ff G 6eZ // ijff 

J ? o; ,X,g p ,W,r 

(1) there exists G 6el and (q,a) =7/ (r,(3~f3) or 

(2) there exists G 6eZ and (g, a) =7 S ( r,(3~/3 ). 

pop transitions (ran, 7eZ, o, i, e) G <5 e with l G Q e \ {ran} if: 

• a - U uV G ^ $ a-A)' g q - e bel and («’ a ) ( r > £ )‘ 

• 7/W occurs in bel, a _ V ^;' g _ G 7 iff there exists a a ’^[’ q ~ G 6eZ and {q,a) =7 g 
(r,e). 

• 7/W occurs in bel, a ^£ q - £ £ iff 

(1) there exists G 6el and (q, a) =$7 (r, e) or 

Oi ,U ,C[ J 

(2) there exists a -’x' g q ~ ^ freZ and (Qi a ) (r, e). 

• 7/W does not occur in bel, a ^ N x q ^ G 7 i/f 

(1) there exists G 6el and (q,a) =7/ (r, e) or 

(2) there exists a -'x’l- £ freZ and (?) a ) =^9 (r-, /5 — /3). 

e-transitions (7, 6eZ, e, run, bel') G <5 e if: 

- 'y-* - £ {/f i/iere exists g an d x ’ r g £. 

a ,X JJ a ,X ct,X,g 


We say that a configuration is stable if its associated state is run. 


Example 5.9. Let us explain the runs given in the Figure 5.5. It starts in the initial 


configuration (run, { lo’u’go } 


which represents the empty run. 

From qo there exists only one path of observation in in the POp 

r 7.U.90 1 
t±n.U.OnJ 


correct, by reading in on the estimate VPA we reach (run, 


-Lo.U.go 
r -Lq.u.to i 
l-Lo.lW 


IPA. As this path is 
). The new element 


of the stack { _Lq U u 9 *? 0 } means that the stack of the possible run has head 7 and its current 
state is qo after a correct run, moreover the run entered q 0 when it pushed this 7 and 

it does not have a second non-terminal element in our stack. Reading a second in is 

r 7,u.go i 

L W.U.On J 


still doable by a single run, we reach (run, 


7,U,<?o 
r 7.U,go i 
t±0,U,g 0 7 
f -Lo,U,go i 

t I n.U.nr ,J 


which modifies one information 


- J-o,U,go - 

compared to before: we know from the bottom part of the head stack that the stack has 
at least a second 7 . 
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Reading a serve then is possible as there exists a correct signalling run from q$ to 
q\ with observation serve. The estimate VPA modifies the head stack so as to represent 
that the run we follow is now in q\ but without modifying anything else. 

Reading a pop event is more involved: from q\ with head of stack y, triggering a 
pop can be done by a correct run staying in q\ or by a faulty run going in f±. To 
represent this and the popping of the stack, we go in two steps. In the first step, we go 
to the state {} which keeps the information of the two possibilities of cur¬ 
rent configuration and we pop the stack. In the second step, we deterministically take 
an £ transition that transfer this information from the state to the stack. In order to 
transfer the information, the estimate VPA checks which of the current possible runs 
(represented by and yyfy^) corresponds to each of the new head of stack. This 

is done by comparing the bottom part of the run with the top part of the head of stack, 
here 7 , U,go in, every cases. Reading a second pop realises a similar process reaching 
(run, { jlo’u’go > if go } )• ^ n empty would lead to (run, {7717®} ) as there is a cor¬ 
rect run from q\ to qo labelled by empty but no run from f± with such label. Similarly a 
reset cannot be taken from q\ but it can be read from f\, thus we reach (run, { 77777 } )• 

The following proposition links runs of V and observed sequences of A(V). 

Proposition 5.1. Let a be an observed sequence of A(V) and p* be its correspond¬ 
ing finite run with successive stable configurations (run,wo)... (run,w n ). Let w n = 
beli ■ ■ ■ belh and for i < n, bel W be the top stack symbol of Wi. Then: 

• For all 'I'yyb'i'qb 1 ^ belh, there exists a sequence ( 1 )o <i<h such that for 

all i, ^ anc [ a signalling run p of V such that V(p) = cr that reaches 

configuration (qh,yi ■ ■ . 7 h)- In addition: 

• ifX h = U then p may be chosen correct; 

• if Xh 7 ^ U then p may be chosen faulty; 

• if X), = W then there exists 0 < k < n, such that p^ is faulty and W does not 
occur in bel^ k ~ l \ 

• Conversely, let p be a signalling run of V such that V(p) = o reaching configu¬ 
ration (qh, 71 ... 7 ft), there exists a sequence (— y)o<i<h such that for all i, 

- 7 i y Xi,9i - £ belt. In addition: 

7<—i)Xf— 1,®—1 1 

• if p is correct then X/, = U; 

• if p is faulty then 7 ^ U ; 

• if there exists 0 < k < n, such that p±^ is faulty and W does not occur in bel 
then Xh = W. 

The difficulty of this proof is the number of cases that have to be studied: what is the 
tag and which kind of transition (local, push or pop) is considered. As a consequence, 
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we only detail the most involved case (when the event is a pop and the tag is W. The 
result is obtained by induction on the size of the observed sequence and mostly consists 
in understanding the definitions of A(V) and especially of the tag updates. 

Proof. We prove the result by induction on |<r|. The basis case is straightforward. For 
the inductive step, we only detail the most involved case: a[n] £ X 0 + For the properties 
related to tags, we only detail the ones related to W. Denote o' = <r[l]... <j[n — 1] and 
w n -1 = bel\ ... bel' h bel' h+l . 

• Let -— 7 fe y X '‘’ 9 '! - £ belh- By construction, there exists fh±l^eySh±i g bel' hA _, with 

7h-i.Xh—i i n J ’ Uh’X-h’Qh h+1 

p" 'y' X' q' 

7 h = 7 h, a signalling run (q' h+1 , y' h+1 ) =t> (q h ,e) with proj{p") = a[n], , h - \ — £ 

bel/ h where ( 7 fe-i> X^, g^-i) = (y h -i,^h-i,qh-i) and X h is obtained by updating X' h+1 
w.r.t. bel' h+1 and p". In particular if X/ ( = W then: 

(1) X ' +1 = W, or 

(2) W does not occurs in bel' h+l and (a) X ^ +1 = V or (b) X! h+l = U and p" is faulty. 
By inductive hypothesis, there exists a sequence (—— L, *’ i , — )o<i<h such that for all i, 

, ^ and a signalling run p' of V such that V(p') = o' reaching configu- 

ration (q' h+ 1 ; 7 i.. • 7 ^+i )• Consider the signalling run p = p'p 77 ; it reaches configuration 
(, 7 j .. .y'h). Since for all i < h. bel\ = 6 eZ*, the sequence (- 7 — tb v i , — )o <%<h and 
the run p are appropriate. The three additional properties follow from the rules of tag 
updates. In particular, if Xh = W, then: 

o the assertion ( 1 ) holds and then the property comes from the inductive hypothesis, or 
o the assertion (2) holds which implies that W does not occur in bel' h+l and p is faulty. 

• Let p be a signalling run of V such that V(p) = c which reaches configuration 

p" 

(q h , 71 ... 7 /,). Let us write p = p± n -ip" with {q' h+1 ,y' h+1 ) =7 (qh,e)- By the in- 
ductive hypothesis, there exists a sequence (—— tb i ’ ‘, — )o<i</i+i such that for all i, 

, ~ Y X ' lf - £ bell and for all i < h, 7 - = 7 + By construction, , lh ^f h ' qh , — g for 

some X;, . Since &eZ* = Z>eZ( for all i < h, we obtain the required sequence of items. 

The three additional properties follow from the rules of tag updates. In particular, 
assume there exists 0 < A; < n, such that pp ; is faulty and W does not occur in bel k ~ l . 
o If pin -1 is correct then, as p is faulty, p" is faulty and W does not occur in beP^ 1 = 
bel'h+i- So by construction Xh = W. 

0 If P\.n-i is faulty then: 

• either X! h+1 = W and by construction Xh = W, 

• or X' h+ j = V. By induction hypothesis there does not exist 0 < k < n — 1, 
such that pik is faulty and W does not occur in bel k ~ 1 . So W does not occur in 
ber- 1 = bel’ h+1 . Therefore X h = W. 

□ 
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2.2 Decidability of diagnosability for POpVPA 


Our goal in this subsection is to use the characterisations from the Section [3] of Chap¬ 
ter [3] to decide the diagnosability of POpLTS generated by POpVPA. To do so, we 
face the difficulty that the Borel sets that characterise IF-, IA- and FF-diagnosability 
are not a priori regular, even in the finitely-branching case. Yet, for POpVPA, we cir¬ 
cumvent this problem, and manage to specify these sets by pLTL formula on a product 
of the POpVPA with its estimate VPA, the tags are used to define the atomic propo¬ 
sitions. The decidability of the qualitative model checking for recursive probabilistic 
systems |IEY12| then yields the decidability of the above three diagnosability notions 
for POpVPA. 

The first step is to build the product of the POpVPA and its estimate VPA. From 
this point on, we assume that the set of states of the POpVPA is separated between 
correct and faulty states Q = Q C U Qf. This can be done without loss of generality: 


the transformation ensuring this is similar to the one shown for pLTS in Section 1.4 
of Chapter [2| We build Vq(y) = V X A(V) the product automaton of V and A(V) 
synchronised^on the alphabet of observed events T, 0 . The transitions of V labelled by 
unobservable events do not change the second component of the state and the transitions 
of M(V) labelled by £ do not change the first component of the state. Due to the 
determinism of A(V), V_4(y) has the same probabilistic behaviour as the one of V except 
that it memorises additional information along the run. More precisely, let p be a run 
of V, then p, a run of Vmy), is obtained from p by following the same transitions and 
adding the single © transition hrable after any pop transition. One immediately gets 
Py_ A ( V) ( / o) = Py(p). Formally we have: 

Definition 5.7. Given (V,V,T 0 ) a POpVPA V = (Q,E,r,<5, P) and its estimate VPA 
A(V) = (Q e , run , E 0 , F e , S e ), their synchronised product is the pVPA V^yj = ( Q A , T, U 
{©^r- 4 , ^P- 4 ) where: 

• Q A = Q x Q e is the set of states with initial state q A = (qo iC ,run); 

• T - 4 = r x T e is the stack alphabet with Tj_ X r^j_ the set of bottom stack symbols 

and Yq 1 = (© 0 ) initial symbol; 

• The transition relation V 4 consists of: 


local transitions. 

• For all (q, 7 , a, q', y') E 5 with a unobservable and bel E T e , we have 
((<?, run), (7, bel), a, (q ', run), (7', bel)) E 5 A ; 

• For all ( q, 7, a, q', 7') E 5 and (run, bel, o, run, bel') E 5 e with V(a) = o, we 
have ((q, run), (7, bel), a, (q', run), (7', bel')) E S A ; 

• For all (l, bel, e, run, bel') E 5 e , q E Q and 7 £ T, we have 

((q,t), (7, bel),Q, (q, run), (7, bel')) E 5 A ; 

push transitions. 

• For all (q, 7, a, q', 7'7") E 6 and (run, bel, o, run, bel'bel'') E 5 e with V(a) = 
o, we have ((q, run), (7, bel), a, (q', run), (7', bel')( 7", bel")) E V 4 ; 
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pop transitions. 

• For all (q, 7, a, q', e) £ 5 and ( run,bel,o,£,e ) £ 5 e with V(a) = o, we have 
({q, run), (7, bel), a, (q', £),e) £ S A ; 

• The transition probability function P' 4 is defined by: 

- P A ({q, run), (7, bel), a, (q', run), (7', bel')) = P(q, 7, a, q', 7'); 

- P- A ((g, run), (7, bel), a, (q', run), (7', bel')( 7", 6eZ")) = P(g, 7, a, tV')/ 

- PYY n/n), (7, bel), a, (q', £),e) = P (q, 7, a, q', e); 

-forte Q e \{run},P A ((q,£),(y,bel),Q,(q,run),('y, bel')) = 1. 


Example 5.10. T/ie product POpVPA contains the current run of the POpVPA and 
the information given by the estimate POpVPA. Let us consider the faulty rim given as 
example in the Figure 5.2 This POpVPA does not follow the separation between correct 
and faulty states. Here we write q c (resp. qp) if the state q was reached by a correct 
(resp. faulty) run. After reading in, we are in state (qo tC ,run) meaning that the state 

of the possible run is qo, it was reached by a correct run and our estimate VP A is in 

r 7iU,go,c i 
L J_n.lJ.an ^ J 


state run, the head of stack is ( 7 , 


-Lo,U,go, 
r -U)iUigo,c i 

L J_n.LI.an J 


meaning our real head is 7 and the rest 


- ±o,U,go,c 

is the head of the estimate VP A. If we follow the faulty run until after the first pop, 
we reach the state (fij, { , ^ 7777 ^}); we are thus in fi with a faulty run and the 

estimate VPA is in one of the temporary states. In order to leave this state, we read 
a © which leads to the state (f if, run ). © is an event only affecting the part of the 
POpVPA corresponding to the estimate VPA, allowing it to realise the e transition that 
follows the observation of a pop event. 


Given a finite run p of V, we inductively define the run p of V_ 4 (y) as follows. First 
(qo, ± 0 ) = (Qq 1 -L(f). Let p of length n > 1, a £ £ and q £ Q and 71 ,..., 7 h £ T such 
that p = p'a(q, 71 ... 7 h)- If a 0 then p = p'a((q, run), ( 71 , belfi)... (jh, bel h )) where 
(run, bel\ .. - belh) is the configuration reached by V(p) in -4(V). If a £ then p = 
p'a((q, £), ( 71 , beli) ■ ■ ■ ( 7 h,bel h )) © (( q,run), (ji,beli)... (jh-i, bel h -i)(y h ,bel' h )) where 
(£, beli ■ ■ ■ belh) is the configuration reached by V(p) in A(V) and (run, beli... belh-ibel' h ) 
is the single next configuration reached by an e transition. As previously observed, 
P(p) = P (p). 

In order to prove decidability of diagnosability for a POpVPA V, one wants to check 
whether the formulae characterising diagnosability defined in Chapter [3] hold on V. Let 
us first recall the relevant results of Chapter [3j We defined three path formulae: 

• f: for every run p, f (p) = true if p is faulty; 

• it: for every run p, it (p) = true if there exists a correct signalling run p' with 

v(p) = v(p'y, 

• 2 U: 2 U(e) = false and 2 U(goao • • • Qn+i) = true if 


firstf(P(g 0 ao • • • (ln+i)) = firstf (V(q 0 a 0 ... q n )) < 00 
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where firstf(cr) = min{fc | 3 p signalling run V(p) = a A p\ k is faulty} with the 
convention that min(0) = oo. 

Using these path formulae, given a pLTS A, we obtained the following results: 

• A is FF-diagnosable iff A |= P =0 (OD(f A 11)); 

• if A is finitely branching, A is lA-diagnosable iff A |= P =0 (OD(il A 2U)). 


The paths formulae f, il and 2B depends on the past of the run and not only on the 
current configuration. We therefore transform the pathL formulae into pLTL properties 
that are checked on Vg(yy First, for each path formula we define an atomic propositions 
on the pairs ((q,run)( 7 , bel )) consisting of a state of together with a top stack 

contents. 


Definition 5.8. Let bel C 2( rxT g x< 2) 2 ', we say that the tag X occurs in bel if there exists 

c bel 

7-,x-,<r oeL ■ 

The atomic propositions ug, v u and u w corresponding to the path formulae f, il and 
2 B are defined by: 


• Vf((q,run)( 7 , bel)) = true if and only if q G Qg; 

• l ' u ((q,run)( 7 , bel)) = true if and only if U occurs in bel; 


• v w ((q,run)( 7 , bel)) = true if and only if\N occurs in bel. 

We extend vg, n u and v w over configurations cf = ((q,£),w)) with t 7 ^ run by 
Vf{cf) = v u ( cf) = is w ( cf) = true. 

The atomic propositions ug and v u perfectly reflect the paths formula f and il, and 
v w is eventually forever true if and only if 2IJ is. 


Proposition 5.2. Let p be an infinite run ofV. Then: 

• For all k G N, f(pgk) v f (\ast(p ik )) and il (pyk) <=> ^ (last(/Opt)); 

• p |= OD2B <t4 3K\/k > K. i/„(last(pj,fc)) = true. 

The second component of V_ 4 (y) representing A(V), one can use the results of Propo¬ 
sition 5.1 to link the tags and the runs associated with the observed sequence. This is 
what we do here. The most complicated (and interesting) case being the link between 
2IJ and W. The idea is the following. When the tag W disappears after following an 
observation in A(V), let n be the observed length of the last time W was not tagging 
any state, then the oldest fault in the current run occurred after the n’th observation. 
Thus every time W is not present, the longest prefix of the run that is surely correct 
increased, ensuring that 2D is false. Of course, 22J can be false more often than the 
absences of W. However, if after the n’th observation, for n £ N, W always tag a state 
of the belief, it means that there exists a run consistent with the observation for which 
a fault occurred at most at the n’th step. Therefore firstf is bounded, which implies 
that 2B will eventually become forever true. 
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Proof. First, note that f and vj obviously coincide: they both express that a fault 
occurred. 

To prove the second item, about it and u u , we use the link between observed sequences 
and the tag U in Vg{y) . Let u be an observed sequence triggered by a run of V. Then bel a 
is the top stack symbol of the stable configuration in A(V) reached by the run accepting 


c (so ending by an e-transition if the last event is a pop event). Due to Proposition 5.1 
U occurs in bel a iff there is a correct signalling run of V with observed sequence a. 
According to the definition of v u , we thus deduce that for any finite signalling run p of 
V, i/ u (last(p)) = true iff it (p) = true. 

We now establish the link between 2H and . To show the left-to-right implication, 
let p E D and Kq E N be such that p,Ko |= LIST. By definition of 211, firstf (7* (p,|,fc)) 
is constant and bounded by Ao for k > Kq. For all k E N, let bel^ be the top stack 
symbol reached in A(V) after reading the observed sequence P(pik)- If for a ll k > K 0 , 
W occurs in bel} tl then for all k > Ao, ^(last(pifc)) = true. Otherwise there exists 
K\ > Ao such that W does not occur in belx 1 ■ Let k > K\. as firstf (fP(pykf) < Ao, 
there exists a faulty run p' of V^y) such that V(p') = P(pyn) and p\k 0 faulty. W 

W occurs in bel *.. 


5.1 


does not occur in bel^x and p'^ Kl+1 is faulty. Thus by Proposition 
Therefore for all n > K] , i/ UJ (last(p| n )) = true. 

Let us show the right-to-left implication. Let p E D and K E N be such that for all 
k > K, v w (\ast(pik)) = true. By definition of v w for all k > K, W occurs in belk 
(dehned as above). Let k > A, by Proposition 


5.1 


there exists a run p of Vg(y) such 
that V(p') = V(pik) and there exists n < k such that p'^ n is faulty and W does not 
occur in bel n -\. Thus n < K. Therefore for all k > K, firstf ('P(pifc)) < K. Since 
beyond K, firstf is bounded, it is non decreasing and then ultimately constant. Let K' 
such that for all k > A', firstf (V(p^k)) = fi r stf(7 3 (/9^/ c _i)). So p, A ' j= DST and thus 
p |= OD2H. ;□ 


Thanks to the relationships between the path formulae, and the atomic propositions, 
and using the characterisations from Section [3] of Chapter [3j we reduce the FF-, IF- and 
lA-diagnosability to the model checking of a pLTL formula on the product VPA Vg(yy 
Model checking qualitative pLTL for probabilistic pushdown automata is doable in 
polynomial space in the size of the model |EY12| . In our case, V^y) is exponential in 
the size of V. We thus obtain the decidability and a complexity upper-bound for the 
diagnosability problems for POpVPA. 


Theorem 5.3. FF -diagnosability, IF -diagnosability and I A- diagnosability are decidable 
in EXPSPACE for POpVPA. 


Proof. Thanks to the Propositions |5.2| and 5.1 and the characterisations of Proposi¬ 


tions 3.7 (page 80) and 3.8 (page 81), we can derive pLTL characterisations of diag¬ 
nosability for POpVPA. Namely, for V a POpVPA, as V and V^y) have the same 
probabilistic behaviour, 


• V is FF-diagnosable iff V_ 4 (y) |= P 0 (O□(i'/ A v u ))] 

• V is lA-diagnosable iff V^y) |= P =0 (OD(i/ u A v w )). 
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Moreover, since POpPDA have finitely many states, the POpLTS they generate are 
finitely-branching. Therefore, I F-diagnosability coincides with FF-diagnosability accord¬ 
ing to Theorem 3.1 The two above qualitative pLTL formulae can be checked on proba¬ 


bilistic pushdown automata thanks to jEY09| . More precisely, one can transform Vg(y) 
into a recursive Markov chain (the transformation is linear) JEY12I . Then, the model 
checking of qualitative pLTL on recursive Markov chains is doable in PS PACE in the 
size of the Recursive Markov Chain and EXPTIME in the size of the formulae |EY09| . 
In our case, the product VPA V^y) is exponential in the size of V and the size of the 
formulae is constant. This yields an EXPSPACE algorithm for checking diagnosability 
of P Op VPA. D 


2.3 EXPTIME-hardness of the diagnosability for POpVPA 

While the notions of diagnosability we studied in the previous section are decidable 


in EXPSPACE (Theorem 5.3), this is not necessarily optimal. Here, we only show an 
EXPTIME lower bound on the complexity. This lower bound is obtained by reducing the 
universality problem for VPA, which is known to be EXPTIME-complete (AM04| . This 
reduction also applies to FA-diagnosability for which the decidability status is unknown. 

Theorem 5.4. FF-, IF- ; FA- and IA -diagnosability are EXPTIME -hard for POpVPA. 

Proof. Let us start with FF-diagnosability. The proof is done by reduction from the 
universality problem for VPA, which is known to be EXPTIM E-hard |AM04] , Recall the 
universality problem for VPA: given a VPA A and a set of final states Qf, do we have 
V{{p G SR | last(p) G Qf}) = £“? 


Starting from a VPA A we build a pVPA V' (see Figure 5.6) with two components: 
one correct and one faulty, both reachable in one step from the initial state. The correct 
component is a copy of A with a positive probability of making a reset (emptying its 
stack and going back to the initial state of A) in a final state. Every reset starts by a 
new observable event t),followed by some pop event b and ends by a second t|. In the 
faulty component, one can read any observation of X* and also has the possibility to 
produce \ and b in a way that mimics a reset. If a \ is read, then the faulty component 
triggers some b and a \\ as would be done in the correct component. This way, the 
observation associated with a reset does not give any information on the correctness 
of a system. What matters is after which observed sequence can a reset occur. If an 
observed sequence cannot end in an accepting state of A. then in a faulty run, with 
probability 1 this observed sequence will be read in between two resets, revealing the 
fault. Reciprocally, if A is universal, everything that can be observed on a faulty run 
can also be observed in the correct component establishing that V is not diagnosable. 

Formally, from a VPA A = (Q, X,r, 5) and a subset of accepting states Qf QQ, 
we build a pVPA V' = {Q', X', T', S', P') as follows: 

• Q' = Q U {/o, /b, q' 0 , q\>} and q' 0 is the initial state; 

• X' = XW {f, u, b,t)}; 
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• r' = ri±i{#} and r ± = r ± ; 

• Writing 5^ resp. 5$ and 5 \, for the set of local resp. push and pop transitions of 
V, S' consists of the following transitions: 

local ht, U {(q' Q , ± 0 , u, ±o, q 0 ), {q' 0 , ±o J,-bo, fo),(fo, f\>) I 7 € T U {_L 0 }} U 

{(<?,7>tl,7,®) I ? 6 Qf,7 £ T U {_L 0 }} U {(/o,7)°)7,/o) I a G Xt,,7 G 
{B, -Lo}} U {(<?b, -Lo, t], T 0 ,®), (A, -l-o, t], -Lo, /o)}; 
push <5 # U {(/o, 7 , a, 7 -B, /o) | a 6 £#,7 € {-B,T 0 }}; 

pop <5 b U {(f 0 ,B,a,£,f 0 ) | a G X b } U {(/ b , £, b, e,/ b )} U {(<? b , 7 , b, e, g b ) | 7 ^ T}; 

• P 7 is such that for every 7 E T, P 7 (/o, 7 , t], 7 , / b ) = and assigns arbitrary positive 
probabilities to the other transitions in S'. 

We further consider the POpVPA (V',E 0) ?) with X 0 = X U {b, t|} and the masking 
function satisfies V{u) = V(i) = e and V(x) = x for any other event x G £/ This 
construction is illustrated in Figure |5.6| The figure uses the following shortcuts: a b G £ b , 
G £[,, a# G £jj, 7 G T, 7 ' G {S, T 0 } and zGT\ {-L 0 }. 



Figure 5.6: A POpVPA for the EXPTIME-hardness of FF-diagnosability. 


The observed sequences corresponding to correct runs in (V',T, 0 ,V) are either of 
the form Wi l 0 l \> kl t\uj2 ■ ■ ■ tjb^- 1 ! ]w n or of the form w\\\\> kl \\W2 ■ ■ ■ 't\w n -\\\\) m . In these de¬ 
compositions, Wi, for i < n, is a sequence corresponding to a run of V starting in q$ 
and ending in some accepting state qf G Qf, ki is the number of elements in the stack 
after reading Wi in V and also in V' (apart from the bottom stack symbol To), w n is 
the sequence associated with a run of V starting in qo, and m is at most the number of 
elements in the stack after reading w n -\ in V. Note that ki only depends on Wi, and 
does not depend on the exact run over Wi, since V is a VPA. 

The observed sequences corresponding to faulty runs in (V/ £ 0 , V) are less constrained. 
They are of one of the two forms presented above, however the words Wi for i < n can 
be any word of £*. 

Let us show that V is not universal if and only if (V 7 , £ 0 ,P) is FF-diagnosable. 
First assume that V is not universal. Then there exists a word w G X* such that no 
run of V reading w ends in an accepting state qf. However, the observed sequence of 
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any faulty run almost-surely contains the factor \\w\\. Indeed, faulty runs almost surely 
visit infinitely often the configuration (/[,,_Lo), and from there, the probability A to 
read \\w\\ is positive. Let p be an infinite faulty run. Its observed sequence is of the 
form V(p) = wi\\\> kl \\W 2 ^> k ' 2 \\W‘i ... with ki < \wi\ for every i. If there exists i < n such 
that Wi = w then p is surely faulty, since it has no corresponding correct run. The 
latter statement can be refined. For n > |rc|, if, for every i < n, \wi\ < n and there 
exists i < n such that W{ = w then p^rd+n is surely faulty. Indeed, < 2n + 1, 

w occurs at the latest for i = n, and once it occurs the prefix is surely faulty. Let 
us therefore consider faulty runs that do not satisfy this property. We let Avoid n = 
{p G F | V(p) = wi\\b kl \\W2 , 0 l \> k2 \]W3 ■ ■ • A (Vi < n Wi / w V 3i < n \wi\ > n)}. By 
construction, FAmb2 n 2 +n C Avoid, „. Moreover, using standard union-sum inequalities, 
P(Avoid n ) < (1 — A) n + Tp: (recall that A is the probability to read \\w\\ from (/q,_Lo)). 
Thus linin-^oo P(Avoid n ) = 0 and hence limjj-^oo P(FAmb n ) = 0 so that (V 1 ,Ti 0 ,V) is 
F F- di agnos able. 

Assume now that V is universal. Let p be an infinite surely faulty run of (V',T, 0 ,V). 
We write p' for the greatest ambiguous prefix of p and a G X D U {t|, b} such that p'a is 
again a prefix of p. Observe that a cannot be b since the number of b’s between two 
tj’s, whether on the left or right-hand-side of V , is entirely determined by the word of 
E* read before the first For the same reason, if a = t], V(p') ends with a word of E* 
(i.e. the number of b’s in V(p') is even). Let w be the greatest suffix of V(p') contained 
in £*. If a = t], we deduce that there is no run starting in qo with observed sequence 
w and ending in an accepting state of V. Therefore, V is not universal. Similarly, 
if a 6 E 0 , then there is no run starting in q q and with observed sequence wa. In 
that case also, V is not universal. We hence conclude that there is no infinite surely 
faulty run in (V / ,E 0 ,'P). As the probability to generate faulty runs is positive, this 
implies that (V / ,£ 0 ,'P) is not IF-diagnosable. Now, I F-diagnosability is equivalent to 
FF-diagnosability for finitely-branching POpLTS (see Theorem 3.1), and so (V / ,E 0 ,'P) 
is not FF-diagnosable. 

Let us now argue for the EXPTIME-hardness of FA-diagnosability and lA-diagnosability. 
The reduction is very similar to the previous one: it only requires an additional state 
in the correct component that ensures that almost surely any correct run will be iden¬ 
tified as being correct. Therefore the problem may only come from the faulty runs 
which are dealt with exactly as above. From the VPA V = (Q, S,r, S) and pVPA 
V' = (Q', S', T', 5') defined above, we construct a pVPA V" = (Q", X", T", 5", P ,r ) such 
that 


• Q" = Q l U {q c } and q' 0 is the initial state; 

• S" = S U {f,u, ft,a}; 

• r" = r ; 

• 6" = 5' U {(t?,a, 7 ,c?c) I 7 € ru{l 0 },? G Q U {<? c }}; 

• P /; assigns arbitrary positive probabilities to transitions in 5". 
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We further consider the POpVPA (V",H 0 ,V) with E 0 = £" \ {f,u}, and the masking 
function satisfies 'P(f) = V(u) = e and V(x) = x for any other event x. The construction 
is illustrated in Figure [5T| where we use the shortcuts: a\> G £|,, G Xj,, aj G Sjj, 7 G T, 
7 ' G {B, J_o} and zGT \ {J-o}. 




Figure 5.7: A POpVPA for EXPTIME-hardness of FA-diagnosability and IA- 
diagnosability. 


V" is a slight modification of V 7 : from any state of V (accepting or not), reading 
the new letter a leads to the sink state q c . As a consequence, for any correct run of 
(V", X 0 , V ), there is a positive probability at each step to perform event a and become 
surely correct. This implies linin^oo P(CAmb n ) ng pj = 0. Observe that the above proof 
for V also applies to V": V is not universal if and only if (V // , X 0 , V) is FF-diagnosable. 
Now, since lim n _>.oo P(CAmb n ) ng N = 0 , FF-diagnosability, FA-diagnosability and IA- 
diagnosability coincide for (V 7/ , T, 0 ,V). We conclude that V is not universal if and only 
if (V", T, 0 ,V) is diagnosable (for any notion of diagnosability). □ 


While diagnosability was undecidable for POpPDA, the situation is more complex 
with POpVPA. Some notions become decidable, although we may not have the ex¬ 
act complexity, while for others decidability remains open. The method used here 
requires a characterisation that can be translated in pLTL. This is not possible for FA- 
diagnosability or the approximate notions of diagnosability which cannot be expressed 


in pathL. Recall that pathL is more expressive than LTL according to Proposition |3.6 
page [79 


3 Diagnosability of infinite pLTS represented by stochastic 
Petri nets 


In this section we study infinite-state pLTS generated by stochastic Petri nets (SPN). 
This model is incomparable to pPDA and therefore generates different kinds of pLTS. 
In Subsection 3.1 we formally define SPN and the infinite-state pLTS generated by an 
SPN. We then show in Subsection 3.2 that, as for pPDA, diagnosability is undecidable 
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in SPN, and that a restriction similar to what pVPA are to pPDA is not enough to 
regain decidability. 


3.1 Stochastic Petri nets 


A Petri net contains places and transitions. In each place there are tokens and a 
transition consumes tokens from input places and produces tokens in output places. 
From a manufacturing point of view, these tokens can be seen as different items that are 
received, assembled with other items, processed and the final product can be exported. 
Due to the locality of transitions, Petri nets are appropriate for modelling concurrent 
systems. The infinite behaviour comes from the potentially unbounded number of tokens 
inside the net. The sets of infinite-state pLTS that can be generated by pushdown 
systems and Petri nets are incomparable. 

Definition 5.9. A Petri net (PN) is a structure N = (P, Mo,T, Pre, Post), where P 
is a set of m places; Mq is the initial marking, i.e. a vector M : P —> N that assigns to 
each place of a PN a non-negative integer number of tokens; T is a set of n transitions; 
Pre : P x T —> N and Post : P X T —> N are the pre- and post- incidence functions 
that specify the arcs. We also define C = Post — Pre as the incidence matrix of the net. 


For Mat E {Pre, Post, C) and t E T, we write Mat(-, t) for the column vector which, 
for every i E N, contains at the row i the value Mat(z, t). A transition t is enabled from 
M iff M > Pre{■ ,t) and may fire yielding the marking M' = M + C{- ,t). One writes 
M [cr) to denote that the sequence of transitions a = tj 1 - ■ ■ tj k is enabled from M , and 
M [cr) M' to denote that the firing of cr yields M'. One writes f E cr to denote that a 
transition t is contained in cr. The length of the sequence a (denoted |cr|) is the number 
of transitions in the sequence, here k. 


Example 5.11. Consider the PN of Figure 5.8 The initial marking is Mq = [2, 0, 0, 0,0]. 
Two tokens are needed to fire t 2 , one in place p\ and one in place p 2 - In order to take 
the manufacturing analogy again, this means two items must be assembled here. In p\ 
two items are already here, ready for assembling, however p 2 is empty. Firing t\ de¬ 
livers one item to p 2 , enabling the transition 1 2 - Once t 2 was fired, one token in p\ 
and one token in p 2 are consumed and one token (the assembled product) is produced 
in p 3 . There two transitions can be fired. For example, the token can be consumed by 
ti producing a new token in p± which enables t.Q. This last transition consumes a token 
without creating any new one, so it could correspond to the finished product being sent, 
and thus removed from consideration by this system. A sequence of transitions corre¬ 
sponding to the arrival of new products, their processing and removing from the system 
is a = toPOOie- Firing this sequence uses exactly the tokens that are created inside it. 
Therefore it can be repeated: a k is enabled from Mq for all k E N. 


The set of all sequences that are enabled at the initial marking AIq is denoted L(N), 
i.e., L(N ) = {a E T* \ Mo[cr)}. A marking M is reachable in N iff there exists a firing 
sequence cr such that Mq [<t) M. The set of all markings reachable from Mq defines 
the reachability set of N and is denoted R(N). Given k E N, a place p of a PN N is 
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Figure 5.8: A Petri net. Circles are places and rectangles are transitions. In the initial 
marking, p\ has two tokens represented by the two black dots. 


k-bounded if for all M £ R(N), M(p) < k. It is bounded if there exists k E N such that 
p is fc-bounded. A PN is bounded (resp. ^-bounded) iff all of its places are bounded 
(resp. fc-bounded). 


Firing t± k times in Mq leads 


Example 5.12. Consider again the PN of Figure 5.8 
to the marking M\ = [2, k, 0, 0, 0]. Therefore the place P 2 is not bounded. 


Probabilities are added to a PN by adding a fire rate to every transition in the 
following way. 


Definition 5.10. A Stochastic Petri Net (SPN) is a pair AT = (N,p) where N is a PN 
and for all t E T, p(t) £ M + is the rate of firing of transition t. 


The usual interpretation of rates is that, in a given marking, a delay is computed 
for every enabled transition t with an exponential probability distribution function of 
parameter pit), i.e. the probability distribution function for the delay of transition t is 
ft : x E M + i —y p(t)e~^^ x . Multiple time semantics [ HM09 ] can be chosen in an SPN to 
decide how these delays are used to determine which transition is fired. For instance, 
one could use (a) a single server policy: each transition can only be fired once by a given 
marking, (b) a race policy: the transition whose firing delay elapses first is assumed to 
be the one that will fire next and (c) a resampling memory policy: at the entrance in 
a marking, the remaining delays associated with all transitions are forgotten. Observe 
that as we use an exponential probability distribution, whether the delays are forgotten 
or not does not modify the probabilistic semantic. Using these choices, one could then 
define the semantics of the SPN as a continuous time Markov chain. However, as we 
only focus on discrete-time semantics here, we simplify the definition of the probabilistic 
behaviours of the SPN. We remove the time consideration from the semantics, and only 
keep the discrete time Markov chain induced by the continuous time Markov chain. 
This semantics keeps enough information to answer questions expressed for example by 
pLTL or pCTL formulae, but cannot address time-related issues such as mean reaction 
time. 

Using the simplified interpretation of rates, as for pLTS, a probability measure can 
be defined on the sequences of transitions of a PN. Given a sequence a E T*, we 
write C(a) for the set of infinite sequences prefixed by a, C{a) = {a' E | 3a" E 
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T w : o' = aa”}. The set of infinite sequences is the support of a probability measure 
defined by Caratheodory’s extension theorem from the probabilities of the cylinders: 
the probability of the cylinder starting by the empty sequence e is equal to 1 and, for 
at a sequence, the probability of C(at ) in Mq, written P(<rt), satisfies 


P (at) = P(cr) X 


[ct') p(t') 


As for pPDA, we enhance SPN with a mask function. 


Definition 5.11. A partially observable SPN (POSPN) is a tuple (J\f, £ 0 , V) consisting 
of an SPN A f equipped with a mapping V : T —> £ 0 U {e} where £ 0 is the set of 
observations. 


From now on, we assume that there does not exist a marking M reachable from 
Mo and an infinite sequence a E T u such that V{a) = e and M[a). This assump¬ 
tion corresponds to the assumption of convergence that was made for pLTS. The ob¬ 
served sequence w of observations associated with the sequence a is w = V{a). Note 
that the length of a sequence a is always greater than or equal to the length of the 
corresponding observed sequence w (denoted \w\). Given a word w E L*, we write 
P(w) = Yl (J &P~ 1 (w) T(fr)- Thanks to our earlier assumption, this sum is finite. 

Example 5.13. Consider again the PN N of Figure |5-4 We define the POSPN 
{(N, fi),{a,b,c},V) such that for all t E T,p(t) = 1 and V(to) = V{t\) = b,V(t 2 ) = 
a. V(t:i) = V{t/f) = £ and V{t§) = 'P(tg) = c. The observed sequence bac corresponds 
to the sequences t^tsts and t^ttfe, each of which has a probability Therefore 
P (bac) = 

The (potentially infinite) pLTS associated with a POSPN is based on the reachability 
graph of the PN: every state corresponds to a reachable marking. 

Definition 5.12. A POSPN (J\f,Yi 0 ,V) defines a pLTS Ay = (Qjy,Mo,T,Tjy,Pjy) 
where: 


• Qn = A(Mq); 


• T s pn = {(M, t, M') | M[t)M'}; 


• For every (M, t,M') E Tjy, P jy[(M,t,M')\ 


5Zt'6T,M[i'> UP) 


This pLTS is infinite when the reachability set is infinite. This happen iff the PN is 
not bounded. If the PN is fc-bounded, for k E N, then the size of the generated pLTS 
is exponential in the size of the PN and in k. A POSPN is diagnosable according to a 
notion of diagnosability if the pLTS it generates is diagnosable. 

In order to mirror the POpVPA restriction of POpPDA, we introduce the notion of 
visible POSPN. 


Definition 5.13. A visible POSPN (VSPN) is a POSPN such that 
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• an unobservable transition does not modify the number of tokens in the system; 

• for every pair of two transitions t\ and t _2 with V{t\) = V(t 2 ) Post(- ,ti)l v — 
Pre(■ ,ii)l„ = Post{■ ,t2)f v ~ Pre{■ ,t 2 )f v where l v is the vector with 1 in every 
position. 

This second condition means that the number of tokens is modified similarly by t\ 
and t 2 - An observer of a VSPN thus knows at all time how many tokens are present in 
the system. 

3.2 Undecidability of diagnosability for stochastic Petri nets 

The exact diagnosability problems for fc-bounded Petri nets are decidable as the gen¬ 
erated pLTS is exponential and the exact diagnosability problems for finite pLTS are 
decidable. Moreover deciding if a Petri net is bounded is also decidable |Rac78| . For un¬ 
bounded Petri nets however, while non-stochastic variants of diagnosability are known 
to be decidable on Petri nets, this is not the case for the stochastic notions of exact di- 
agnosability. In order to show the undecidability, we reduce the problem of the language 
inclusion for Petri nets, namely: given two PN ./V 1 and IV 2 , an observation alphabet X 0 
and a mask function V does V(L(N 1 )) C V(L(N 2 )) hold? This problem is known to 
be undecidable(see the survey (EN94| ). 

Theorem 5.5. The FF-, IA- and FA- diagnosability problems of POSPN are undecidable. 

Given two PN ./V 1 and N 2 , we build an SPN where the initial transition (which can be 
faulty), produces tokens in one among two components. This component corresponds to 
an enhanced copy of one of the two given PN. Then, a sequence of this PN is triggered. 
At any moment during this sequence, a transition starting a reset operation can be 
taken. This reset operation removes all the tokens from the PN then produces the 
tokens corresponding to the initial marking so that a new sequence can be read by the 
PN. The goal is that the fault is detected iff an observed sequence that can only be 
triggered from N 1 is observed in between two resets operation. 

The difficulty of this reduction lies in the reset operation as one cannot test directly 
whether the places of the PN were correctly emptied. This information however, can 
be encoded in the observation. Let us now describe what happens in a reset. The reset 
starts and ends by an observable (1 and, in between, produces a certain number of b. 
Each of these b removes a token that was left inside the PN, so that at any moment, the 
observer knows precisely the number of tokens within the system. If there is still at least 
one token in the system when the second Jj occurs, a gadget is used to allow the system 
to trigger any observed sequence so that no information is given to the observer. In 
other words, the observed sequence in between two reset operations give an information 
on the system iff the previous reset had correctly emptied the PN. 

Proof. Let N 1 = (P 1 , Mg, T 1 , Pre 1 , Post 1 ) and N 2 = (P 2 , Mg , T 2 , Pre 2 , Post 2 ) be two 
PN, with the mask function V and the observation alphabet E 0 . 
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Without loss of generality, we assume that the initial marking Mg has a single token 
in a place p l 0 for i = 1,2, that every transition is observable and that there exists an 
integer k E N such that the number of tokens in the system is equal to k times the 
length of the sequence plus 1. This last assumption could be ensured by choosing k as 
the maximum number of tokens added by a transition (this means number of tokens 
created minus number of tokens consumed) and adding an additional place where the 
unnecessary tokens are put {i.e. if a transition adds k' tokens with k' < k, then this 
new place receives k — k' tokens). 



Figure 5.9: Reduction from language inclusion. The Figure 5.10 represents the content 
of the box Box 1 , it is similar for Box 2 . Transitions are labelled by their name and their 
observation. 


We build the POSPN (P, Mq,T, Pre, Post, p,T, 0 U (represented in Fig¬ 


ure 


5.9) where: 


P = P 1 u P 2 u {po} u { p ; 


emp 


i Prum Perr 


* = 1 , 2 }; 


• T — T 1 U T 2 U {tj n , f^,V rese , t l resn | * — 1) 2} U {t l a | a E E, i — 1, 2} U {temp, t P rr 
p € P 1 U P 2 }; 


• for i E {1,2 },p E P\t E T\Pre(p,t) = Pre l (p,t) and Pre(p l run , t) = 1, 

Pre(po, tj n ) 1, -Pre(p em p, f j) 1, Pre(p run , ^resn) — 1 • -^^^-{Perri^rese) ^• 

for a E E, Pre{p l err , f a ) = 1, Pre{p,t p em p) = Pre(p l emp , t p emp ) = 1, Pre(jp,t p er r) = 
P re {plmp^err) = 1. When undefined, Pre(p,t ) = 0; 

• for i E {l,2},p E P\t E T l , Post(p,t) = Post l (p,t) and Post(p l run ,t ) = 1, 
Postal, t\ n ) = Post(p l run ,t\ n ) = 1, Post(p l run ,t\) = Post(p l 0 ,tl) = 1, for a E 
E, Post(p l err , f a ) = l, Postal, t l a ) = k, Post(pl mp ,t l resn ) = Post(p l emp ,t l rese ) = 1, 
Post{p l emp ,t P emp) = 1, Post (pg, t P rr ) = 2, Post(p\ rr , t p err ) = 1. When undefined, 
Post(p, t ) = 0; 

• V extends V on N by, for p E P 1 U P 2 ,i E {1,2}, a E E, V'(t\ n ) = Jt, P'(tJ) = 
Viferr) = V (t\. ese ) = V {t{ esn ) = fj, V {tf a ) = a, V (t P emp ) = b; 

• for i E {1,2}, p E Pi, p{tj e8e ) = p{t l resn ) = p{t p emp ) = 2/c(|E| + |Ti|) (assuming 
|Ti| > 1) and for every other transition t p[t) = 1. 
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Figure 5.10: Content of the box Box 1 . 


Moreover, tj = f is the fault transition. 

We show that the system is FF-diagnosable iff V(L(N 1 )) % V(L(N 2 )). 

First note that the set of observed sequences associated with the infinite sequences 
starting by the transition t\ n , denoted L l , contains exactly the words of the form 
... u;fcj}b nfc j}... where for all 1 < j < k, (1) wj G £*, (2) YjL=l k \ w m\ + 1 > 
Ei=i n m and (3) Emil k\ w m\ + 1 = J2m=i n ™ implies Wj G V(L{N 1 )). 

Suppose that V{L{N 1 )) C V(L(N 2 )). Let a be an infinite faulty sequence. As 
a is faulty, it initially fired tj n , thus V(a) G L 1 . Thanks to the above remark on 
the languages Li, and as V{L{N 1 )) C V(L(N 2 )), V(c t) G L 2 , therefore there exists 
a sequence a' starting by the transition tf n with same observation as a. Moreover 
this transition is not faulty as it did not fire t\ initially and cannot fire it after the 
first transition. Therefore V(cr) is not surely faulty. As this is true for every faulty 
sequence, the system is not IF- diagnosable. The pLTS generated by this POSVN 
being finitely branching, according to Theorem |3.1[ this implies that the POSPN is not 
FF-diagnosable. 

Suppose now that ^(^(A^ 1 )) % V(L(N 2 )). There thus exists a word w such that 
w G V{L{N 1 )) \ V(L(N 2 )). The observed sequences of L\ such that there exists jGN 
with Em=i k\ w m\ + 1 = Em=i Um an d w i = w are sur ely faulty as they do not belong 
to L 2 . We denote SL\ the set of these observed sequences. Let us show now that with 
probability 1 an infinite faulty sequence belongs to SL\. 

While a token is in p\ rr or p ] run , every transition taken with observation other than 
[j produces k tokens in the copy of N\. Moreover, there are at most |£| + |Ti| such 
transitions, each with rates 1. As the transition triggering (j has rate 2/c(|£| + |Ti|), the 
expectation of the number of tokens produced before a J) is below 1 + 2 k. During a reset 
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operation, as for all places p G Pi, p(temp) = 2fc(|£| + |7i|) and the two transitions 
producing jj have rates 1, the expectation of the number of token removed from the 
copy of Ni, assuming there are enough tokens within the system, is greater than 1 + 
fe(|S| + |Ti|Q Thus, with probability 1, a faulty sequence will infinitely often remove all 
the tokens from P 1 . Therefore with probability 1, the observation of an infinite faulty 
sequence will be of the form ■ • • rcfc(jb nfe (j • • • G L 1 with infinitely many i £ R 

such that Ylm= l k\ w m\ + 1 = Ylm= l n m- There is a probability p > 0 that for any such 
i, Wi = w as w G P(L(A r1 )). Therefore with probability 1, there exists i £ N such that 
Wi = w. Hence with an infinite faulty sequence almost surely belongs to SL\. This 
implies that the POSPN is IF-diagnosable and thus FF-diagnosable. 

In order to reduce the problem to FA-diagnosability and lA-diagnosability, we pro¬ 
ceed similarly to the proof of Theorem |5.4[ we add another place p c and a transition 
t c that takes a token from p% un and puts it in p c . This transition has firing rate 1 and 
observation \\. This is thus the only transition with this observation. As a consequence, 
taking this transition ensures the run is surely correct and remains that way. As a run 
entering Box 2 almost surely infinitely often contains a token in Pr Unl every run almost 
surely either becomes faulty or surely correct. Therefore lim n —^ CAmb n = 0. This 
implies that in this POSVN, FF-diagnosability is equivalent to FA-diagnosability and 
lA-diagnosability. The rest of the proof above then applies. J2t 

An interesting feature of this proof is that the number of tokens in the POSPN used 
in the reduction can be deduced from the observation at all time. Therefore it is a 
VSPN. This gives the following result. 

Corollary 5.1. The FF-, IA- and FA- diagnosability problems of VSPN is undecidable. 

Thus, a restriction similar to what allowed us to regain decidability in POpPDA is 
not enough for POSPN. 

4 Conclusion 

The study of diagnosability for infinite-state pLTS depends heavily on the model used 
to finitely represent such a pLTS. Choosing a model that is too powerful leads quickly 
to undecidability. This has been shown with the undecidability proofs established for 
restricted classes of POpPDA and POSPN. These proof contains important differences. 
For instance, while undecidability is proven for every notion of stochastic (and in fact 
even non-stochastic) diagnosability in POpPDA, it is only proven for the exact notions 
of diagnosability in POSPN. Moreover, it is known that non-stochastic diagnosability 
is decidable in PN. In this sense, PN is a model for which there is still hope to get 
decidability results. We did not use the notion of coverability graph here, which gives a 
finite over-approximation of the reachability graph. Maybe an analysis of its language 
coupled with a study of the pathological behaviours (due to the over-approximation) 
may help in solving A FF-diagnosability. Moreover, the restriction that was used for 

1 This value is obtained by analysing the case where only one place contains tokens. 
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POSPN was chosen to mimic the one used on POpPDA, but is not necessarily the most 
suited to the model. 

Even if the POpPDA model has the strongest undecidability results, the appropriate 
restriction allows to regain decidability. For POpVPA, we could use model-checking 
methods to verify pLTL formula equivalent to the logical characterisation of Section |3]of 
Chapter [3j This only gave decidability of the notions for which a logical characterisation 
was known. Many questions are still left open partially as a consequence. 

First, it would be interesting to find ways to close the complexity gap between our 
upper and lower bound for the decidable diagnosability notions. The complexity of the 
current decision procedure comes from an exponential determinisation and the use of a 
PS PACE model-checking result. As we are interested in specific simple formula, there 
may be a way to verify them in PTIME instead. The exponential of the determinisation 
seems harder to remove. 

Second, we would want to determine the decidability status of FA-diagnosability. If 
it is undecidable, it would confirm the difference in complexity with the other notions of 
exact diagnosability that the logical characterisations showed. However, this difference 
was shown for infinite-state pLTS in general, not for pLTS generated by POpVPA. It 
is possible that, as for finite systems, FA-diagosability could be decided for POpVPA 
with the same complexity as the other exact diagnosability notions. 

Finally, one may be interested in considering the case of the approximate diagnos¬ 
ability notions. The method used here cannot be applied. Moreover, it is unclear now 
what the POpVPA restriction simplifies for approximate notions of diagnosability. Re¬ 
call that in the finite case, no determinisation were used to solve these notions, allowing 
for a PTIME algorithm. We conjecture that this notion remains undecidable. 
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Chapter 6 

Control of the degradation in 
probabilistic systems 


Embedded systems are often equipped with one (or more) controller(s) that can modify 
the behaviour of the system in reaction to the environment. Controllers can, for exam¬ 
ple, be used in order to maintain some vital functionalities of the system when facing 
a failure of a component. As controllers need to detect failures to react efficiently, it is 
tempting to add to controllers a diagnosis task. In other words, the system will contain 
some choices that can be made and which will alter the behaviour of the system while 
satisfying its specification. Controllers will then resolve these choices in order to render 
the system diagnosable. Controllers can be formalised in multiple different ways. For 
example, controllers could be within the system and thus have have full knowledge of the 
behaviour of the system or they could rely on partial observation similarly to diagnosers. 
Since the goal is for controllers to deduce the existence of a fault, we cannot assume 
they know exactly the state of the system, and thus it must rely on partial observation. 
Formally, some of the observable events are controllable and, considering its current 
observation, the controller chooses which subset of events the system can trigger. A 
system is then said to be actively diagnosable if there exists a controller ensuring its 
diagnosability. In |SLT98| , the authors showed that the active diagnosability problem is 
decidable in doubly exponential time for non-probabilistic systems. Then in 11111MS17| . 
the authors designed a single exponential time algorithm and proved this complexity to 
be optimal. In the probabilistic case, the controllable system can be represented by a 
weighted transition system in the active case. This weighted transition system, coupled 
with a controller, produces a pLTS that can have infinitely many states (depending on 
the memory required by the controller). Thus, unsurprisingly, the active probabilistic 
diagnosability is more complicated than the corresponding passive problem: exact diag¬ 
nosability is PS PACE-complete in pLTS (see Chapter [fj while it is EXPTIME-complete 
(see [BFH + 14]) for controllable weighted LTS (a controllable variant of pLTS). 

However the choices performed by the controller ensuring active diagnosis may have 
a pernicious effect: to detect faults, controllers sometimes could favour the occurrence of 
these faults! Forcing a fault in the system easily ensures diagnosability but contradicts 
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the initial goal of trying to maintain important functionalities of the system. Additional 
requirements can thus be made to controllers in order to manage the degradation of the 
system. Thus, a controller ensures safe active diagnosability if the controlled system is 
diagnosable and there is a positive probability that an infinite run is correct. In other 
words, the controller is allowed to increase the probability of a fault in order to ensure 
diagnosability, however it must maintain a positive probability of correct behaviours. A 
quantitative version of this requirement fixes a threshold e to the probability of correct 
runs that the controller must achieve. Unfortunately, safe active probabilistic diagnos¬ 
ability is undecidable |BFH + 14) . However, when limited to finite-memory controllers, 
the problem becomes decidable in NEXPTIME |BFH + 14) . Safe active diagnosability 
may be too strong a requirement for some real systems. Indeed, systems age and what¬ 
ever control is applied, their components will eventually fail. Thus, in many cases, the 
fault can be considered unavoidable by the system. As a consequence, some systems are 
designed to behave correctly for a long period of time at the end of which they will be 
replaced by a new system. Instead of trying to force runs to stay correct, a controller 
could try to slow the speed at which the system fails. This expresses a different kind of 
requirements for the degradation control of a system. We formalise the framework and 
these requirements in Section [l] establishing a few semantical results along the way. 
Then, in Section [2. 1| we present the algorithmic results. 

This chapter develops and extends some of the results from |BHL17bj . 


1 Degradation of a probabilistic system 


In this section, we give formal definitions of the degradation of a system. These degra¬ 
dation notions have to be satisfied by the system simultaneously to diagnosability, en¬ 
suring that any fault is detected and the system does not produce faults too often or too 
quickly. As this combination depends on the notion of diagnosability chosen and our 
focus here is more on degradation, we only use FF-diagnosability (which is the simplest 
notion of exact diagnosability that we introduced in Chapter [2]). 

In terms of observation, we use in this chapter a partition between observable and 
unobservable events (see discussion of Section 1.3 of Chapter [2]). 

In Subsection E3 we give the definitions of degradation for pLTS. Then, in Subsec¬ 


tion 1.2 we show how to add a form of control and state the problems we are interested 


m. 


1.1 Degradation in passive systems 

When protecting a system from degradation, we want that it has a sufficient probability 
not to trigger a fault, or at least, that if a fault has to occur, it can be postponed as 
much as possible. We study different notions of the degradation of a system: safety, 
fault freeness and resiliency. 

A pLTS is safe [BFH^14] if it guarantees a positive probability of infinite correct 
runs. A pLTS that is not safe is thus doomed to trigger a fault with probability 1. The 
probability to stay correct could however be arbitrarily low. So we can quantify the 
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notion in order to refine it: for e > 0 , a pLTS is e-safe if this probability is greater or 
equal to e. 

Definition 6.1. Let A be a pLTS, e > 0 . A is e-safe if P(C 00 ) > e. It is safe if 
P(Coo) > 0 . 

As pointed out in the introduction, in some cases, safety is a too strong requirement. 
We formalise now two alternatives: fault freeness and resiliency. Fault freeness aims 
at quantifying the period of time during which the pLTS is correct. We introduce 
a discount factor 7 < 1 on duration in order to vary the importance given to the 
length of the correct runs. When 7 is chosen small, only the beginning of the runs 
matter. This focus on the short-term is useful for systems that are regularly replaced 
for example. A greater 7 will on the opposite be chosen if one wants the system to be 
correctly performing for a longer time. The expectation of this discounted value is then 
compared to a threshold v. 

Definition 6.2. Let A be a pLTS, 0 < 7 < 1 and v E [ 0 ,oo]. 

• A is (7, -y)-fault free ifYl n > i^(Cn)7 n > v. 

• A is lasting fault free if it is (1,00) -fault free. 

Clearly, for any fixed value of 7, the greater v is, the better the system. Remark 
also that for 7 < 1 , the sum Xm>i IP(Cn)7 n is finite and smaller than For 7=1, 
Xm>i IP(Cn)7 n is the mean observable length of the maximal correct signalling prefix 
of a random run, which can be infinite. This justifies the name lasting fault free when 
the expectation is infinite. 

The notion of resiliency is an alternative measure of degradation based on a factor 
of degradation ratio per time unit a < 1 . A pLTS is a-resilient if the proportion of 
finite correct runs which stays correct on the next occurrence of an observable event is 
asymptotically greater than a. This requirement has two qualitative variants: strong 
resiliency (resp. weak resiliency) requires a-resiliency for every (resp. for at least one) 
a < 1 . In other words, a system is weakly resilient if asymptotically, the probability 
to be in a correct run of observable length n is greater than an exponential a n . And 
a system is strongly resilient if this probability is asymptotically greater than all such 
exponential. 

Definition 6.3 (Resilient pLTS). Let A be a pLTS. 

• Let 0 < a < 1. A is a-resilient if lim sup n ^. oc 5^77 = 0 ; 

• A is strongly resilient if for all 0 < a < 1, A is a-resilient; 

• A is weakly resilient if there exists 0 < a < 1 such that A is a-resilient. 


Example 6.1. Let us consider the pLTS A of Figure |7T7} We give examples of the 
different notions of degradation by studying some choices of probabilities (pi)ieN- 
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f , 1 - Pi ! f , 1 - P2 f, 1 - P3 



Figure 6.1: An example of infinite pLTS with parametric probabilities (pi)i<=N- 


A has a single correct run p = qoaq\aq 2 ..with observation aF while every faulty 
run contains an infinite number of ‘b’. A is thus FF -diagnosable. Moreover, the prob¬ 
ability of p is U n >iPn an d the probability of its prefix of length n is r n = Y\ i<n Pi- 
Consequently, A is safe iff liirin^^ r n > 0. This can be achieved by choosing pt = 1 — i 
for example. 

Also, by direct application of the definition, A is lasting fault free iff Yln>i r n = °°- 
Let us consider different values of ( Pi)i 

• Let pt = tAj. Then r n = Thus A is not safe but is lasting fault free. For every 

a < 1, lim n ^. 0O (n + 1 )a n = 0. Thus A is also strongly resilient. 

• Let pi = u+yp- Then r n = Tfppyp ■ ThiLS A is neither safe nor lasting fault free. For 
every a < 1, lim n _ 5 . 00 (n + l) 2 a n = 0. Thus A is strongly resilient. 

• We inductively define two sequences mk and by: 

nk = mj (hence no = 1) and rrik = nk + Yljck m j + n j- 

We also define the intervals: 

• h = [nk + Y.j<k m j + n jt nij + nfi; 

• Jk = [J2j<k m j + nj, n k+ i + Y,i<k m j + n A- 

When i € Ik, we choose Pi = \. When i 6 Jj., we choose Pi = 1. 

Observe that for all n 6 Jk, r n = 2~ mj . Consequently 

r " = ^2^0<k mj 2~^<k m j = oo. 
n> 1 /c>0nGJfc k> 0 

Thus A is lasting fault free. 

Let k € N and n = Ylj<k m j + n j- Consequently, r n = 2~ . Fix a = 

C/ n 

— = )-Jdj< k m j + n 3 > 2 mfc (v / 2 ) _2mfc = 1 . 

r n 

Therefore A is not a-resilient. 

The next theorem establishes the precise links between the qualitative versions of 
the three degradation notions for pLTS. 
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Theorem 6.1. Let A be a pLTS. 

• If A is safe then A is lasting fault free and strongly resilient; 

• If A if finite then: 

A is safe iff A is lasting fault free iff A is strongly resilient; 

• There exists a lasting fault free pLTS that is not strongly resilient; 

• There exists a strongly resilient pLTS that is not lasting fault free. 

The first assertion is quickly obtained from the definitions and the last two come 
directly from the previous examples. The second one requires a bit more development. 
As the pLTS A is finite, one can use the notion of bottom strictly connected component 
used in Chapter [4] to characterise the diagnosability notions for finite pLTS. A notable 
difference is that, in Chapter [X] we had to consider the BSCC of an enriched pLTS. Here, 
we show that every notion of degradation is equivalent to the existence of a reachable 
correct BSCC of the pLTS A. 

Proof. Let A be a safe pLTS. There exists e > 0 such that for all n, P(C n ) > s. Thus, 
E „>1 IP(Cn) > En>i e = oo. Moreover, for all a < 1, lim^oo < lining ^ = 0. 

Thus A is both lasting fault free and strongly resilient. 

Let A be a finite pLTS. Observe that every BSCC of A contains either only correct 
states or only faulty states. Accordingly we can speak of faulty BSCC or correct BSCC. 
As A is a finite pLTS, we know that almost surely an infinite run reaches a BSCC and 
that the mean time to reach a BSCC is finite (see e.g. |BK08| ). Due to the first result, 
A is safe iff there exists a reachable correct BSCC. 

Suppose that A is not safe. 

• Every reachable BSCC are faulty which implies that the mean time to reach a faulty 
BSCC if finite. This mean time is an upper bound on the mean observable length of 
the maximal signalling prefix of a correct run. Thus A is not lasting fault free. 

• We note m = \Q\. For all q E Q c , there exists p q a run starting in q composed of an 
elementary run from q to a faulty BSCC followed by an elementary run (or circuit) in 
the BSCC of which only the last event is observable (by convergence). This run has an 
observable length smaller or equal to m. We note p q , the probability of that run and 
p = min (?g Q c p q . Consider a signalling run p of observable length n for an arbitrary n and 
ending in q E Q c . From the existence of p q , P({p' E SR n+m nC | p A p'}) < (1 — p)P(p). 

Th 1 

Thus P(C„ +m ) < (1 - p) P(C„). So, P(C n ) £ 0((1 - p)™)- Choosing a = (1 - p)™, A 
is not a-resilient and thus not strongly resilient. □ 

1.2 Controlled systems 

Extending the pLTS formalism in order to express control requires to fix at least two 
features of this formalism: the nature of the control and the distribution of probabili¬ 
ties of the controlled system. Intuitively, we want the control and the diagnosis to be 
realised by the same device: from its observations, it restricts the system in order to 
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diagnose it and limit its degradation. The control is thus done with partial observation. 
So we recall the Controllable Labelled Transition System (CLTS) from jBFH + 14 . In 
this model, in order to specify the control, a subset of observable events is considered 
controllable. The controller forbids a subset of controllable events depending on the 
sequence of observations it has received. Thus the controller cannot modify its choice 
between two observations. The transitions of the system are no more labelled by (ra¬ 
tional) probabilities but by (integer) weights which measure their relative possibility of 
occurrence. Given a state and a set of forbidden controllable actions, the weights of 
the transitions exiting this state and labelled by uncontrollable or allowed controllable 
actions are normalised to obtain a probability distribution. If the controller does not 
introduce any deadlock, the controlled system is a live pLTS. 


Definition 6.4. A Controllable Labelled Transition System (CLTS) is a tuple C = 
(Q,q 0 ,T,,T) where: 


• Q is a set of states with an initial state qo £ Q; 


• X = X Q tt) Ti u is a finite set of events partitioned into the set of observable events 
X D containing controllable events X c C X D and the set of unobservable events X u 
containing the fault f ; 


• T:QxSx(3->N is the transition function that associates an integer weight 
with each transition. 


A CLTS induces a labelled transition system which transition relation is defined by 
q q' if T(q,a,q') > 0. The extended relation =>• and the other usual definitions are 
defined as for pLTS. We assume that the CLTS is convergent and live. 


a a 


P b ^ f P 



a, b 


Figure 6.2: An example of CLTS. Weights are all equal to 1 and omitted on the figure. 
The only controllable event is b. 


Example 6.2. A CLTS C is represented in Figure 6.2. If the control enables every 
event, the run qouqiaq\bq 2 has probability 1/8. If the control always forbids ‘b’, this 
same run has probability 0. And if it only allows ‘b ’ after observing one ‘a it has 
probability 1/4. 
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We now formalise the ingredients necessary to define how to control CLTS. Let 
S' C E and q € Q, let us write G s * (q) for the sum of the weights of the transitions 
exiting q and labelled by an event of EV Using this sum, we define a normalisation of 
the transition relation restricted to the events of E* by: 

T e ' („, «,„') = ( W if ° € E ‘ a,,d GE ' <«)>* 

0 otherwise. 


A strategy of a CLTS C is a function n : E* —Dist(2 s ) such that for all w £ E* and all 
E* £ Supp(7r(u;)), E\E C C E*. In other words, given an observation, a strategy chooses 
(possibly with randomisation) a set of allowed events that contains the uncontrollable 
events. Let C be a CLTS and it be a strategy, we consider the configurations of the 
form (w, q, S*) £ E* X Q X 2 s with w the observed sequence, q the current state and 
E* the set of allowed events by 7r after observation of w. We inductively define the set 
Reach 7r (C) of the reachable configurations under 7r by: 

• for all E* £ Supple)), we have (e, go,£*) £ Reached); 

• for all (w, q, E*) £ Reach^C) and all a £ E. u such that q A q' , we have (w, q ', E*) £ 
Reach 7r (C), and the corresponding transition is denoted by (w, q, E*) Att (w, q', E*); 

• for all (w,q, E*) £ Reached), all a £ E 0 n E* such that q A q' and all E* 7 £ 
Supp(7r(u;o)), we have (wa , q' , E* 7 ) £ Reach 7r (C), and the corresponding transition 
is denoted by (w, q, E*) (wa, q' , E* 7 ). 

A strategy 7r is called live if for every configuration (• w,q,T !*) £ Reach 7r (C), we have 
( q ) 0. Only the live strategies are relevant as the other strategies create deadlocks. 

We are now in a position to introduce the semantics of a CLTS controlled by a live 
strategy 7r in terms of a live pLTS. Its set of states is Reach 7r (C) augmented by an 
initial state to randomly choose the initial control according to 7r(e). The probability 
distributions are based on T s * if the current control is E* combined with the random 
choice of 7r in case of an observable event occurrence. 


Definition 6.5. Let C be a CLTS and it be a live strategy, the pLTS C n induced by the 
strategy ir on C is defined by C 1r = (Q n , S, qo n , T n , P^-) where: 

• Q-jy { qoir } C Reach 7r (C), 


for every (e, q 0 , E*) £ Reach^-(C), (q^, u, (e, q 0 , E*)) £ T n ; 

for every (w, q, E*), (io r , q', E* 7 ) £ Reach,,-^), 

((w, q, S’), a, (w f , q', E* 7 )) £ T n iff ( w , q, E*) (w r , q' , S* 7 ); 

for every (e, q 0 , E*) £ Reach^-(C), P^(qon, u, (e, q 0 , E*)) = 7r(e)(E*); 

for every ((w, q, S'), a, (w, q', E*)) £ T n and every a £ E U; 

P n ((w, q, E*), a, (w,q',YT)) = T s * (q,a,q'); 
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for every (^(w, q, £*), a, ( wa , q ', E**)^ G T n and every a £ E 0 n E*, 
Ptt ({w,q, S’), a, = T SJ '(q,a,q') ■ ir(w.a)(T, 9 '). 


Example 6.3. Consider the CLTS C depicted in Figure 6.2. There are two possible 
enabled subsets: E and E \ {6} t/iaf we denote E~. Let us define the strategy n by 
ir(a n ) = p n ■ E _ + r n ■ E with p n + r n = 1 for all n £ N and ir(w) = Is otherwise. The 
generated pLTS C n is infinite. A part of it is represented in Figure \6.£\ Let ils develop 


the distribution of probabilities exiting the configuration (s,qi,T,). The two transitions 
exiting q\ are enabled with equal probabilities, thus normalised to 0.5. Since ‘a’ and 
‘b’ are observable, the new control is chosen, in the case where a ‘a’ is observed, by a 
probabilistic choice pi-T,~ +ri-E while if a ‘b’ is observed, there is a deterministic choice 
Is- This results in three transitions with probability O.bpi, 0.5ri and 0.5 respectively. 



Figure 6.3: An example of controlled CLTS. 


In Definition 3.2 page 58 we introduced finite-memory diagnosers. Similarly, one 


can formally define finite-memory strategies for CLTS using a set of memory states, a 
memory update function indicating how observations modify the memory state and a 
decision function mapping every memory state to a choice of the strategy. The size of 
the memory is the number of memory states. If the size of the memory of a strategy it 
of a CLTS C is finite, then C n is also finite. 
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Let us define the problems of active diagnosis in the context of the degradation 
control. Roughly speaking, given a CLTS, one asks whether there exists a strategy such 
that the associated pLTS is FF-diagnosable and satisfies the required property related 
to degradation. We distinguish, as usually done, the quantitative problems and the 
qualitative ones (such as safety, lasting fault freeness and strong/weak resiliency). 

Definition 6.6 (Quantitative problems). Given a CLTS C, 0 < e, a < 1, 0 < 7 < 1 
and v € [ 0 , oo]: 

• The e-safe active diagnosis problem consists in deciding if there exists a strategy 
7 r such that C„ is FF-diagnosable and e-safe; 

• The ( 7 , v)-fault free active diagnosis problem consists in deciding if there exists a 
strategy it such that C n is FF-diagnosable and ( 7 , u)- fault free; 

• The a-resilient active diagnosis problem consists in deciding if there exists a strat¬ 
egy it siLch that Cjt is FF-diagnosable and a-resilient. 

Definition 6.7 (Qualitative problems). Given a CLTS C: 

• The safe active diagnosis problem consists in deciding if there exists a strategy ir 
such that C n is FF-diagnosable and safe; 

• The lasting fault free active diagnosis problem consists in deciding if there exists 
a strategy n such that C n is FF-diagnosable and lasting fault free; 

• The strongly resilient active diagnosis problem consists in deciding if there exists 
a strategy n such that C n is FF-diagnosable and strongly resilient; 

• The weakly resilient active diagnosis problem consists in deciding if there exists a 
strategy it such that C n is FF-diagnosable and weakly resilient. 

When tackling problems on strategies, the first step is to wonder if one can re¬ 
strict the strategies that are considered. For example, can we use strategies with finite 
memory. This cannot be done as shown in the following example. 


Example 6.4. In order to illustrate the impact of taking into account infinite memory 

The only ambiguous observed 


strategies, let us examine the CLTS C of Figure 6.2 
sequence is a u . A strategy n thus makes it FF-diagnosable iff the probability of faulty 
rims with this observed sequence in C n is 0. This is done by allowing ‘b’ often enough 
so that it occurs with probability 1. However, the only correct run is p = q 0 u(qia) w with 
observation a u . Thus, C is not actively safely diagnosable. 

Let us denote, as in Example \6.3\ by p n the probability to forbid ‘b’ after the observed 
sequence a n given by the strategy it. Then {q 0 u(o,qi) n ) = ^ n i< n ^Tp L - Thus, by 
choosing p n = 1 — 773 ;, C n is FF-diagnosable, lasting fault free and strongly resilient. 
On the other hand, no finite-memory strategy could achieve this goal since otherwise by 
Theorem\6.1\ C would be actively safely diagnosable. 


Restricting one-self to finite-memory strategies is thus a loss of generality. It can 
however be useful to regain decidability of difficult problems as we will see later. 
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2 Algorithmic analysis of degradation 


In this section we answer the problems listed above by establishing if they are decidable 


and in the positive case by giving their exact complexities. We start in Subsection 2.1 


by proving the undecidability of the quantitative problems. As a consequence, in Sub¬ 
section 2.2 we focus on the qualitative problems and prove the EXPTIM E-completeness 
of all of them except the safe active diagnosis problem. Lastly, in Subsection |2.3| we see 
that the safe active diagnosis problem is more difficult but can still be decided efficiently 
when restricted to finite-memory strategies. 


2.1 Undecidability of the quantitative problems 

The quantitative problems turn out to all be undecidable. The proofs of these results 
are obtained by reductions from the emptiness problem of probabilistic automate^] 

We start by showing the result for the e-safe diagnosis problem, with e > 0. 

Proposition 6.1. The e-safe active diagnosis problem is undecidable. 

The idea of this proof is the following. Given a probabilistic automaton A with 
alphabet S, one builds a CLTS C composed of two independent parts each one initially 
entered with probability \ by an unobservable transition. The unobservable event 
leading to the first part is the fault f which can only be detected almost surely if the 
observable event jj cf E occurs with probability 1. The second part is constituted of a 
CLTS version of A augmented by exiting transitions. One can exit A by allowing a jj. 
When this happens, if the system was in a final state of A it goes to a correct BSCC 
of the CLTS, ensuring the run will remain correct. Else a fault is triggered on the next 
step. This construction ensures the following properties. If there exists a word w with 
an acceptance probability at least 2 e, the strategy which consists in forcing the observed 
sequence w \j ensures a probability of the set of infinite correct runs of at least e. In the 
opposite case, we show that no strategy can achieve this threshold. 


Proof. Let 0 < e < 1/2. We proceed here by reduction from the problem of the 
existence of a word w such that PArc) > 2e. We consider a probabilistic automaton 
A — (Q, Qo >E, (Pa)aeSj F) for which w.l.o.g. we assume that: (1) EH {u, f, jj, jj} = 0 
and (2) the probabilities are fractions ^ with fixed denominator d £ N. One builds the 
CLTS C = (Q', q' 0 , S', T) described in Figure 6.4 and defined by: 

• Q’ = Qc {<2o,<? c ,g/,/i,/2}; 

• S' = EU{f, u, jj, t|}, E u = {f, u} and E c = E U {jj}; 

• the transition function T is defined as follows. 


1. 


Wo f, fi) = W 0 , u, q 0 ) = T(q c , jj, q c ) = T(q f , f, / 2 ) = T(/ 2 , ^ / 2 ) 

= WiJ,/2) = l; 


see page 


115 
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2. for every a G S, T(f 1 ,a, fi) = 1; 

3. for every s,s' £ Q and every a 6 S, T(s, a, s') = d ■ P a (s, s'); 

4. for every s G F, T(s, jj, q c ) = 1 and for every s £ S \ F, T(s, jj, qf ) = 1; 

5. for every other triplet, T is equal to 0. 

As detailed above, the probabilities in A are all multiplied by their common denominator 
d. to obtain integer weights, and we write d ■ A in the figure to represent this scaling. 


M 



Figure 6.4: Reduction to e-safe diagnosability. 


Let us show that there exists a strategy n such that C n is e-safe and FF-diagnosable 
iff there exists a word w accepted in A with probability at least 2e. 

Remark first that, for n an arbitrary strategy, C T is FF-diagnosable iff jj occurs almost 
surely in a run. Indeed, an observed sequence w £ S* is ambiguous. On the other hand 
every faulty run p triggering a jj will produce a \ removing the ambiguity. 

• Assume there exists a word w = a\... ak € X* such that Pu(tc) > 2s. We define the 
deterministic strategy n by: 

• n (w) = {f, u, jj, t]}; 

• for all 0 < i < k, 7 r(oi... a t ) = {f, u, aj+i, t|}; 

• ti(w') = S' for any other word v/. 

Observe that after at most k + 1 observable events, any run leaves Q U {/i} and thus 
t] occurs almost surely implying that C n is FF-diagnosable. Moreover, the probability 
of correct runs with observation tcjjjj is equal to : it is the probability to take u 

initially times the probability to end the observation of w in an accepting state of A. 
As P a{w) F 2e, this ensures that C n is e-safe. 
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• Assume now that for all w 6 X*, P^(u;) < 2.e. Let ir be a strategy such that C n is 
FF-diagnosable, thus with probability 1 an infinite run contains a ft. Moreover, this run 
is correct iff the first ft is followed by a second ft. Then we have: 

IMCoo) = P 7rMtQ 

k;EE* 

\ - ipvM) • p iW 

^ 2 

< e ^ PttM) 
mes* 

= e. 

Therefore, is not e-safe, which concludes the reduction and proves undecidability of 
the e-safe active diagnosis problem. 

We now turn to the ( 7 , u)-fault free active diagnosis problem. It is done once again 
by reduction from the emptiness problem of PA. In fact, it has many similarities with 
the previous proof, but instead of reaching a state q c where the run will stay correct, 
being accepted by the PA only postpones the fault by one step. 

Proposition 6.2. The ( 7 , v)-fault free active diagnosis problem is undecidable. 

The idea of this proof is the following. Given a probabilistic automaton A with 
alphabet X, one builds a CLTS C composed of two independent parts each one initially 
entered with probability ^ by an unobservable transition. The unobservable event 
leading to the first part is the fault f which can only be detected almost surely if the 
observable event ft ^ X occurs with probability 1. The second part is constituted of a 
CLTS version of A augmented by exiting transitions. One exits A with probability ^ at 
every step towards a faulty sub-part except if the ft event is triggered. In this case, if the 
system was in a final state of A it leaves the states of A and postpones the occurrence 
of a fault by one time step compared to if it stayed in A. This construction ensures the 
following properties. If there exists a word w with an acceptance probability at least }, 
the strategy which consists in forcing the observed sequence reft as long as the run stays 
in A ensures an average observable length (without discount) of the maximal correct 
signalling prefix greater or equal to 1. In the opposite case, we show that no strategy 
can achieve this threshold. 

Proof. We proceed here by reduction from the problem of the existence of a word w such 
that P^re) > We consider the probabilistic automaton A = (Q,qo,T,, (P a ) ae v, F) 
for which w.l.o.g. we assume that: (1) X n {u, f, ft, ft} = 0 and (2) the probabilities are 
fractions ^ with fixed denominator d. One builds the CLTS C = {Q ', q' 0 , X 7 , T) described 
in Figure [675] and defined by: 

• Q' = QU {q' 0 , q\, q 2 c , q%, f u / 2 }; 

• X' = XU{f, u, ft, ft}, X u = {f, u} and X c = X U {ft}; 
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• the transition function T is defined as follows. 

1. 

T(q' 0 , f, fi) = T(q' 0l u, q 0 ) = T(qU, q 3 c ) = T(q 3 , #, q 3 ) = T(q 3 , f, / 2 ) 

= T(ql f, f 2 ) = T(/ 2 ,t], / 2 ) = T(fi, (t, / 2 ) = 1; 

2. for every a G £, T(/i, a, /i) = 1; 

3. for every s,s' £ Q and every a E E, T(s, a, s') = d-P a (s, s') and T(s, a, <j^) = 

rf; 

4. for every s G F, T(s, (t, ql) = 1 and for every s G 5 \ F, T(s, (t, = 1; 

5. for every other triplet, T is equal to 0. 

Here again, the probabilities in A are multiplied by the constant d, which we abbreviate 
in the figure by d ■ A. 


M 



Figure 6.5: Reduction to ( 7 , u)-fault free active diagnosability. 


Let us show that there exists a strategy it such that C n is (1, l)-fault free and FF- 
diagnosable iff there exists a word w accepted in A with probability at least 

Remark first that, for n an arbitrary strategy, C T is FF-diagnosable iff \\ occurs almost 
surely in a run. Indeed an observed sequence w G £* is ambiguous. On the other hand 
every run p leaving Q U {/ 1 } almost surely reaches / 2 where tj occurs and, whatever p. 
a fault has occurred. 

• Assume that there exists w = w\ .. . Wk G X* such that > 7 ). We dehne the 

deterministic strategy 7 r by: 

• 7r(rc) = {f, u, Jt, t|}; 

• for all 0 < i < k, tt(iu\ ... Wi) = {f, u, w l+ i, t]}; 
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• 7 r(w') = S' for any other word tv'. 

Observe that after at most k + 1 observable events, any run leaves SU{/i} and thus t] 
occurs almost surely implying that C n is diagnosable. 

By definition of C and 7T, a correct signalling run p such that V(p) = w\ .. .wt for i < 
k has probability ^ of staying correct at the next step depending on if the current state 
is q 2 or belongs to Q. Similarly, a correct signalling run p such that V(p) = w\ ... 
has a probability P^(rc) of being at the next step in q\ and 1 — Pn(tc) in q 2 . Moreover, 
in state qj?, a correct signalling run has a probability \ °f staying correct and in q j? at 
the next step. Therefore for all n £ N, we have n < k implies P(C n ) = {\) n and n > k 
implies P(C n ) = (^-'PaW > (l) n . Finally: i P (Cn) > E~ i(|) n = 1- 

• Assume that for all to £ £*, P/i(u;) < Let 7r be a strategy such that C w is 
diagnosable. Observe that (using a slight and understandable abuse of language): 

r„(c„) = Y, A C) + ^ ^ P 7r ('W(j A C) + E E P,W"AC). 

weY> n uigE n_1 Kk<n uiSS n_fc 

p / c i I 

Let us show that P^(C n + 1 ) < ^ n with a strict inequality if there exists w £ E n-i 

with P^(iytt) > 0. 

IV(Cn+i) = E E F^raAQi E P7r(^tt 2 AC)+ 

weT, n issuftt} we S 1 *- 1 

E E A c ) 

l<k<n weT, n ~ k 

Let us examine the three terms. 

o A correct run p with observed sequence w has a conditional equiprobability that 
last(p) G Q or last(p) = q 2 c . Thus, E„, e s" Ezesul#} ^(wx) = l E«,es™ p wH- 
o A correct run p with observed sequence w \such that k > 1 verifies last(p) = q^. 
Thus, El<fc<n YlweT. n ~ k ^n(wf +1 A C) = \ El <fc<„ T,w&-k ^n(wf A C) 
o A correct run p of observed sequence w Jj has a conditional probability P a(w) that 
last(p) = ql and 1 — P^(ti;) that last(p) = q 2 . Thus: 

E PttH 2 ac)= E PaWp.WaC)^ E p 7rM a c) 

meS"- 1 tueS"- 1 weE 1 *- 1 

with a strict inequality if there exists a word w £ E n_1 with P 7r (t(;(j) > 0. 

By assumption, C n is diagnosable. Thus, according to our characterisation of a 
strategy ensuring FF-diagnosability, there exists a word w such that P 7r (vntt) >0. As a 
consequence, E^=i P (^n) < En^iG)” = 1; thus A is not (1,1) fault free. □ 

Remark 6.1. A straightforward adaptation of the proof shows that for every 0 < 7 < 1, 
A is (7, g37) fault free iff there exists a word w such that P^w) > 
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We end with the a-resilient active diagnosis problem. The construction of the reduc¬ 
tion is a bit simpler. This is due to the fact that the system is FF-diagnosable for any 
arbitrary strategy. In other words, the reduction only relies on the a-resilient property 
to establish undecidability. 

Proposition 6.3. The a-resilient active diagnosis problem is undecidable. 

This time, given a probabilistic automaton A with alphabet X, one transforms A 
into a CLTS, augmented by two states and some transitions. This CLTS is called C 
and its initial state is the initial state of A. At each step, when reading an event of 
X, with probability 1/2 we exit A and will commit a fault in the next step. When a % 
is read after a word wi$... §Wk with for all i < k Wi does not contain (j, either we go 
back to the initial state of A or we will trigger a fault on the next turn depending on 
the probability to accept w^- If a strategy can regularly trigger a word with acceptance 
probability greater than 1 / 2 , it can slow the speed at which the runs become faulty. 


Proof. We proceed here by reduction from the problem of the existence of a word w such 
that P J 4 (w;) > We consider a probabilistic automaton A = ( Q , qo, X, (P a )aeSi F) f° r 
which we assume w.l.o.g. that: (1) X n {rt, f , jj, \\} = 0 and (2) the probabilities are 
fractions with d £ N fixed. One builds the CLTS C = (Q r ,qo,T,',T) represented in 


Figure 6.6 (with some shortcuts to ease readability) and defined by: 


Q’ = QU{<7i,/i}; 

X' = XU{f,tU}, = {f} et X C = XU{«}; 


• the transition function T is defined by: 

1. T(qi,f,fi) = T(f\, t], /r) = 1; 

2 . for every s, s' £ Q, a G X, T(s, a, s') = d • P a (s, s') and T(s, a, q±) = d ; 

3. for every s G F, T(s, (j, So) = 1 and for every s € S\F, T(s , (J, q±) = 1; 

4. for every other triplet, T is equal to 0. 


Once again, the probabilities in A are multiplied by the constant d, which we abbreviate 
in the figure by d ■ A. 

As every fault is followed by a t], whatever the strategy 7 r, C n is FF-diagnosable. 

• Assume there exists w = io\ .. .Wk G X* such that P^tc) > We denote v = P a{w)- 
We define the deterministic strategy 7 r by: 


• tt((?4)*u;) = {f, tl,#}; 

• for all 0 < i < k, 7r((tf;(J)*r(;i... Wi) = {f, \\,Wi + 1 }; 

• ir(w') = X' for any other word w'. 
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Figure 6.6: Reduction to cc-resilient active diagnosability. 


Under strategy 7 r, the observed sequence of a correct run p is some (w$) m w\ .. .Wi with 
0 < i < k. 

o If V(p) = (w$) m wi .. .Wi with 0 < i then with conditional equiprobability, last(p) £ 
Q or last(p) = q\. Thus with probability /. the run will be correct after the next 
observation. 

o If V(p) = (w\.\) m then with conditional probability v, last(p) = qo and with probability 
1 — v, last(p) = q±. Thus with probability v, the run will be correct after the next 
observation. 

Consider an arbitrary n and write the Euclidian division of n — 1 by k + 1 as n — 1 = 
m(k + 1) + i with i < k. One has 2 - ( n-1 )P 7r (C ri ) = (2v) m . Hence = (^)^ fe+1 ^ 

implying lim n _>oo p 2 ^ c ^ = 0. So C n is ^-resilient. 

• Assume now that for every word w £ £*, < \- Let 7r be an arbitrary strategy. 

The observed sequence of a correct run p is some u\$ ... §u m such that for all i. Ui £ S*. 
o If u m ^ £ with 0 < i then with conditional equiprobability, last(p) £ Q or last(p) = q \. 
Thus with probability ^, the run will be correct after the next observation, 
o If u m = £ then with conditional probability P J 4 (u m _i), last(p) = q$ and with proba¬ 
bility 1 -P/i(« m -i), last(p) = q\. Thus with probability P a( u m - 1 ) < 1/2, the run will 
be correct after the next observation. 

Summarising, one has: P 7r (C ri ,) < implying limsup,^^ p 2 ^ c ^ 

So Cn is not ^-resilient. □ 


2.2 Decidability of the Qualitative Problems 

In contrast to the quantitative notions, and to the notable exception of the safe active 
diagnosis problem, all the qualitative problems of diagnosability under degradation 
constraints we introduced are decidable and EXPTIME-complete. The simplest case 
is the one of weak resilient active diagnosability. The proof idea is common to all 
cases: starting from a construction that gives an efficient characterisation of active 
diagnosability (inspired from [BFH + 14] and detailed below), we establish a necessary 
and sufficient condition for the existence of a control strategy that ensures the given 
notion of diagnosability under a degradation constraint. 
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Let us start by defining a construction inspired from [BFH“tl4] [^] This construction 
is the adaptation to the active setting of the FF-automaton used in Subsection 1.1.1 
of Chapter [4j page 93 This construction uses the notion of belief. The initial belief is 
{qo}, and given a current belief B and an observed event 6, the belief obtained after b 
has been observed is defined by: 


A (B, b) = {q £ Q | 3q' £ B , p £ SRi, q' 4> q A V(p) = b} 


A (B, b ) is thus the set of states a partially observable systems may be in, given that 
the previous belief was B and observation 0 occurred. Importantly, it does not depend 
on the strategy as every controllable event is observable. The set of beliefs of a CLTS 
C is denoted BIq and we drop the subscript when there is no risk of confusion. Beliefs 
are of importance since they formalize the discrete information an observer has on the 
current state of the system. Thus, to decide FF-diagnosability of a CLTS, the states 
of the CLTS are enriched with two sets U and V that correspond, respectively, to the 
subset of correct, or faulty, states, that are reachable by a signalling run corresponding 
to the current observed sequence, i.e. to the set of correct, or faulty, states of the 
current belief. Such a pair of sets (U,V) is therefore called a separated belief. As we 
study FF-diagnosability here, one could wonder why we do not only use a set U as we 
did in Chapter [4] In fact, in some of the constructions that we make later, we need to 
know the full belief. For example, forgetting a faulty state could result in a controller 
making a choice that creates a deadlock in this faulty state. By using U and V, we 
have the information pertaining to FF-diagnosis (U) and to the current belief (U UF). 

Formally, from a CLTS C = (Q,qo,T,,T), we define its belief version on the same 
event alphabet C B = (Q B ,q B ,T.,T B ) by: 

• Q b = Q x 2 q x 2 q and qg = (q 0 , {g 0 }, 0); 

• for every (q, U,V) <E Q x 2 Q X 2 Q , for every a £ £, and every q' £ Q 

-if ai X 0 , T B ((q , U, V),a, (, qU, C)) = T(q, a, g')i 

— if a £ £ 0 , letting U' = A(17, a) n Q c and V' = A (U U V, a) n Qf, then 
T B ((q, U, V), a, (q', U', V')) = T(q, a, q'). 

— for every other triplet (( q , U, V), a, (q', U r , V')), T is equal to 0. 

The size of the belief CLTS C B is exponential in the size of C. For the properties we 
are interested in, they have the same behaviour. We introduce 0, a discrete version of 
T b , extended to observed sequences. For w £ £*, (q f , U r , V') £ @((q, U, V),w) as soon 
as there exists a run p such that V(p) = w and (q. U, V ) (</, U', V'). 

We now construct Win the set of all separated beliefs (U, V) such that, starting from 
any (q, U, V ) with q £ U U V, C B is actively diagnosable. This set is computed as a 
greatest fixpoint. We let Wino = 2® c x 2Qf and for n £ N, Win n+ i is the set of the 

2 The difference with the construction of |BFH + 14] is that we focus here on FF-diagnosability rather 
th an lA-diag nosability which simplifies the writing of the proofs. The results of this chapter and 
of fBFH + 14l however hold for both notions of diagnosability. 
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separated beliefs (U, V ) of Win n such that for all state q £ UUV, there exists a sequence 
of sets of allowed events (£*)i <i<k and an observed sequence w = o±... Ok with Oi E S* 
verifying: 

• there exists a run p starting in (q, U, V ) with V(p) = w and reaching (q*, U*,V*) 
with q* E Q c (he. the current state is correct) or U* = 0 (the fault is claimed); 

• Consider a state qi reached from q' E U U V by a run with observed sequence 
o\ ... Oi with 0 < i < k, i.e. (qi, U, V)) E 0((q / , U, V),o\ .. .Oi) for a separated 
belief (Ui, Vj). then: 

1 . the control induced by S* +1 does not create any deadlock: G Si + 1 (qf) 0; 

2. Every new separated belief obtained by an observable step o e£? +1 starting 
in qi belongs to Win n : Vo E V: +l ,V(qo,U 0 ,Vo) E @((qi,Ui,Vi),o),(U 0 ,V 0 ) E 

Win n . 

The computation of Win is in polynomial time in the size of C B , given that in every 
non-terminal iteration at least one separated belief is removed. The correctness of Win 
is established in the richer context of lA-diagnosability in |BFH + 14| . and 7r* a (deter¬ 
ministic finite-memory) strategy ensuring diagnosability consists in, given a separated 
belief (U, V) E Win choosing the greatest set E* such that every possible separated 
belief reached on the next step still belongs to Win. Thus, n* is the most permissive 
strategy ensuring active diagnosability. 

To decide weakly (resp. strongly) resilient active diagnosability, and lasting fault 
free active diagnosability, we build on the belief CLTS construction. 

Theorem 6.2. Weakly resilient active diagnosability is EXPTIME -complete. 

Analysing the set of separated beliefs Win gave a condition for the active diagnos¬ 
ability and in the positive case a deterministic finite-memory strategy n* ensuring it. 
We show in this proof, that in order for a CLTS to be weakly resilient active diag- 
nosable, it needs (1) to be actively diagnosable and (2) C B must contain a reachable 
cycle of correct states associated with separated beliefs of Win. The idea is that if such 
a cycle exists, playing a strategy permissive enough (for example 7r*), there is a fixed 
probability to stay within this cycle and this probability can be used to establish a lower 
bound to the speed at which the system becomes faulty. 

The lower bound is straightforward considering that active diagnosability was al¬ 
ready proven to be EXPTIM E-hard [BFH~tl4j. 

Proof. We first establish the membership in EXPTIME. Given a CLTS C, its belief CLTS 
C B , and the strategy 7r*, we derive a pLTS A. It is obtained from C B by restricting 
it to the states with separated belief in Win and controlled by 7r*. We claim that C is 
actively diagnosable with guarantee of weak resiliency iff there exists in A a reachable 
cycle such that the first component of every state along the cycle is a correct state of C. 
• Suppose first that such a cycle exists in A. We let a > 0 be the probability of this 
cycle, n\ its length, no the observed length of the shortest run reaching a state of the 
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cycle and g the probability of this run. For all n > no, lP^i(C n ) > got' "i . As a 
consequence, A is a'-resilient for all a' < a. A is thus weakly resilient. Therefore, C n *, 
which has the same probabilistic behaviour as A is weakly resilient too. 

• Conversely, suppose that there is no such cycle in A. Let ir 1 be a (live) strategy such 
that C n i is FF-diagnosable. This strategy can be mimicked in C B , ignoring the separated 
belief information. The reachable states of C B are associated with separated beliefs of 
Win (due to the characterisation recalled above). As n* is the most permissive strategy 
ensuring to stay in Win, there does not exist any such cycle in C B either. Consequently, 
there exists n/ G N such that every run p in C B with \p\ > nq ends in a state which 
first component is faulty. Thus Pc , (C n .) = P<jb (C n .) = 0, which means that C n t is not 
weakly resilient. 


The complexity lower-bound is obtained by reduction from the active diagnos- 
ability problem for CLTS, which is known to be EXPTIM E-hard BFH + 14] , From 
C = (Q , qo, X, T) a CLTS, we define the CLTS C = (Q U {q' 0 , q s }, q' 0 , X U {ft}, T') with 
ft a fresh observable event, and such that T'{q' a , ft, qo) = T'(q' 0 , ft, q s ) = T'(q s , ft, q s ) = 1, 
for every q, q' G Q and a G X, T'(q, a, q') = T(q,a,q') and for every other triplet 
T'(q,a,q') = 0. Clearly enough, C' is actively diagnosable iff C is actively diagnos- 
able. Moreover, C is safe by construction, and thanks to Theorem |6.l[ a), it is strongly 
resilient, and thus weakly resilient. □ 


The proof of the next theorem also relies on the set of separated beliefs Win. We 
build a subset of Win, called WinK. A separated belief (U, V) of Win belongs to WinK 
if there exists a strategy it such that from every distribution with support U U V, it 
guarantees to stay in Win, and to give a positive probability to the set of infinite correct 
runs. The CLTS is actively diagnosable with guarantee of strong resiliency iff from 
the initial belief one can reach a belief of WinK while staying in Win. The strategy 
7 T defined with the construction of WinK does not necessarily allows to diagnose the 
system. So the winning strategy consists in cleverly combining the strategy used to 
make the system FF-diagnosable and 7 r. 

Theorem 6.3. Strongly resilient active diagnosability is EXPTIME -complete. 

Proof. Let C be a CLTS. As in the construction preliminary to Theorem |6.2[ we build 
C B , Win and n*. We then define WinKj/ C 2® x Win by a greatest fix point computation. 
For {U', ( U , V) G WinKy , (U, V ) is a separated belief for which there exists a strategy 
allowing to a set of runs starting in U U V to stay in the states of C B associated with 
a belief of Win, and if the run started in U', it stayed correct. WinKjy is obtained 
as the limit of a non-increasing sequence (WinK n ) ng ^ defined inductively by: WinKo = 
{{U', {U, V )) | (U, V) G WinA 0 / F C [/} and for n G N, WinK n+ i is the set of elements 
{JJ ', (U, V)) of WinK n such that there exist a set of allowed events X* verifying: 

• X* does not create a deadlock: Vq G U U V, G s * (q) 0; 

• under the control X* no run starting in a state of U' will make a fault before the 
next observation: \/q c G U' , V p G SRi, q c qr\V{p) G X* =$■ q G Q c \ 
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• every triplet reached by an observable step oGS' belongs to WinK„: 

(U', ( U , V)) G WinK„ with: 

1. U' = {q' c G Q c | 3q c G U[ 1 3p G SRi, q c 4> q' c A P(p) = a}; 

2. U = A(17, o) n Q c and V = A(17 U V, o) D Q/. 

From WinKf/, we define the set WinK C Win by keeping only the second component of 
WinKf/: WinK = {({/, V) G Win | 317', (U 1 , (17, V)) G WinK//}. Let us state some of the 
properties of this construction. 

• By induction, if V)) ^ WinK n then for every (live) strategy and q G U, 

there exists a faulty run starting in q of observable length n; 

• If 0 / U" C U’ then (U',(U,V)) G WinKf/ implies (U",(U,V)) G WinK/r. Thus, 
if (U , V ) i WinK, for all q G 17, ({g}, (17, V")) i WinKf/. 

We also define PreWin the set of states of C B of the form Q x Win from which a 
state ( q , 17, V) with (17,17) G WinK is reachable. Let us show that C is diagnosable and 
strongly resilient iff the initial state of C B belongs to PreWin. 

• Suppose that the initial state belongs to PreWin. Let (U',(U,V)) be an element of 
WinKf/. We define iTmimyp the strategy that ensures to stay in WinK//. This strategy 
immediately derives from the fixpoint definition of WinKf/. For (17, V) G WinK, we also 
define ^(uy) = ^(U'Auy)) f° r an arbitrary U' such that (U', (17, V)) G WinKf/. Finally, 
we let 7To be the following strategy working in three successive phases which may not 
all be triggered. 

1. First 7To mimics n* until a separated belief (17,17) G WinK is reached; 

2. Then, at every observed sequence w, tto chooses to apply 7Tr u,v) with probability 
p w = Tyrgi ■, and to switch to the third phase with probability 1 — p w ; 

3. Finally, 7To behaves forever as n*. 

We observe that C no is FF-diagnosable. Indeed, on the one hand, the events allowed by 
7 To are included in those allowed by the maximally permissive strategy ir*, and on the 
other hand almost-surely, n* is applied from some moment on. Therefore every fault 
will almost surely be detected. 

Moreover, let us prove that it is strongly resilient. Indeed, by definition of PreWin, 
there exists a run p starting in the initial state and reaching a state (q. U, V ) such that 
(17,17) belongs to WinK. Let U' C U the one chosen arbitrarily when defining ityyy 
Without loss of generality, we suppose that p reaches a state of U'. As a fault can only 
be created after p if 7To switches to its third phase, for n > \p\ 0 we have 

n || 

IVo {p ^ | p di p) T 71-0 (p) • 1 = ^7t 0 ( P ) yr • 

'l 1 J- ^ \ -L 

i=\p\ 
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Thus, for every 0 < a < 1, similarly to na n , p a f r , converges to 0. 

• Conversely, suppose that the initial state does not belong to PreWin. Let it be a 
strategy ensuring diagnosability. For every state (q. U, V ) with q £ U reachable by a 
run po with 7 r, (U, V ) 0 WinK and due to one of our observations ({</}, (U, V)) 0 WinK[/. 
Let K be the number of iterations in the fixpoint computation of WinK. Then, for every 
sequence of K random choices under ^ r, there exists a faulty run p £ F, compatible with 
these choices, starting in (q. U, V ) and of observable length smaller than K. Adding up 
the probabilities of runs corresponding to every sequence of choices of 7r we obtain 

G f Ipo| 0 +a' I Po ^ p) > A A|QI P 7 r (p 0 ) 


where A = min g / g q G ^ ql] • Thus, for every n £ N, P 7 r (C ri+jK -) < P 7 r (C n )(l - A^l). 

Letting a = (1 — \ k \Q\)k, we obtain lim n _ > . 0O p " c ^ > 0, so that C 1TQ is not strongly 
resilient. 

To conclude the proof, we observe that the EXPTIM E-hardness derives from the 
same reduction as in the proof of Theorem 6.2 □ 


It turns out that this same combination of strategies can be used to ensure lasting 
fault freeness and FF-diagnosability. In fact, the following theorem establishes that the 
characterisation of the strongly resilient active diagnosability also applies to the lasting 
fault free active diagnosability. 


Theorem 6.4. Lasting fault free active diagnosability is equivalent to strongly resilient 
active diagnosability. 


We show here that the characterisation given in the proof of Theorem 6.3 for a 
CLTS to be actively diagnosable with guarantee of strong resiliency also characterises 
the fact that the CLTS is actively diagnosable with guarantee of lasting fault freeness. 
This shows the equivalence of the two notions in the active case. 


Proof. We reuse the definitions from the proof of Theorem 6.3 Let us show that C is 
actively diagnosable with guarantee of lasting fault freeness iff the initial state of C B 
belongs to PreWin. 

• Suppose that the initial state belongs to PreWin. Then, as discussed in the proof of 
Theorem 6.3, C Wo is diagnosable and there exists a finite run p such that P (p £ C n | p A 
/5)>P(AS- Tims: 


]Tr(c 


«> E 


£ Cn I P S P) > P(P)W E 


1 


n + 1 


= oo. 


71=1 7l=|p| 7l=|p| 

• Conversely, if the initial state does not belong to PreWin. Let 7 r be a strategy ensuring 
diagnosability. For every n £ N, P(C n+ /<) < P(C„)(1 — Thus: 


OO OO -J 

^P(C n ) < K^2( 1 - \ K -W) n < K.\Qb\ 


71=1 


71=1 


\K.\Q\ 


o 


< oo. 











186 


Control of the degradation in probabilistic systems 


Given the equivalence of strong resiliency and lasting fault freeness, from Theo¬ 
rem [G3] we derive: 

Corollary 6.1. Lasting fault free active diagnosability is EXPTIME -complete. 


2.3 Safe active diagnosis problem under finite-memory strategies 

Contrary to the other qualitative problems, safe active diagnosability is known to be 
undecidable |BFH + 14] , In order to regain decidability, one can restrict the strategies 
so that they only use finite memory. Note first that decidability is not immediate 
even if the strategies are assumed to be finite-memory, since no a priori bound on the 
memory is knowrj^] This restriction was studied in |BFH + 14j where the authors give 
an NEXPTIME algorithm. However, the known lower bound is only EXPTIME, leaving 
a gap. We refine here this complexity result by proving that safe active diagnosis can 
be solved in EXPTIME when restricting to finite-memory strategies. 

To do so, we prove a more general result in the context of a well-known model, quite 
popular in artificial intelligence and more recently in formal methods, that combines par¬ 
tial observation, probabilities and control, namely Partially Observable Markov Decision 
Processes (POMDP) |A65| IKL098) . We establish that the existence of finite-memory 
schedulers that ensure a Biichi objective with probability 1 and a safety objective with 
positive probability in a POMDP is decidable in EXPTIME. We then reduce the safe 
active diagnosis of a CLTS C restricted to finite-memory strategies to the existence of a 
finite-memory scheduler in a POMDP Me ensuring at the same time a Biichi objective 
with probability 1 and a safety objective with positive probability. 

Definition 6.8. A partially observable Markov decision process (POMDP) is a tuple 
M = (Q, qo, Obs, Act, T) where 

• Q is a finite set of states with qo the initial state; 

• Obs : Q — > O U {e} assigns an observation 0 £ O to each state. 

• Act is a finite set of actions; 

• T : Q X Act —> Dist(Q) is a partial transition function. Letting Ena(g) = {a 6 
Act | T(q,a ) is defined} the set of enabled actions in state q, we assume that: 

— for all q E Q, Ena(g') A 0, and 

— whenever Obs (q) = Obs (q r ), then Ena(q) = Ena (q') and slightly abusing our 
notation, we denote by Ena(O) the set of events enabled in every state with 
observation 0. 


A decision rule of a POMDP is a distribution from Dist(Act) that resolves one non¬ 
determinism choice by randomization. A scheduler for a POMDP maps histories of 
observations to decision rules. Formally, a scheduler is a function t : 0 + —> Dist(Act) 


3 In the case of Proposition 


4.9 


page 


120 


the restriction in fact made the problem more difficult. 
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such that for every Oi • • • 0j, Supp(r(Oi • • • Oj)) C Ena(Oj). Given a scheduler r, a 
POMDP M yields a stochastic process. This stochastic process can be represented by 
an infinite state pLTS, denoted M(r) in which states are histories of observations. One 
denotes by P^(Ev) the probability that an infinite observed sequence of Ev is realized 
in this pLTS. 

Similarly to what was said for strategies, one can define finite-memory schedulers 
for POMDP. The notion of belief can be adapted to POMDP. As for CLTS, it is a non¬ 
empty set of states that represents the current state estimate, i.e. the set of states the 
system may be in, given the actions (which affect the reachable set of states contrary 
to what is done for CLTS) and observations so far. The initial belief is {go}, and given 
a current belief B , a decision rule 6 and an observation 0, the belief obtained after <5 
has been applied and 0 has been observed is defined by: 

A (B, (<5, 0 )) = |J Supp(r(g, a)) n Obs^O) . 

q£B, a6Supp(<5) 

Aiming at providing a POMDP M<j for the safe active diagnosis problems of a CLTS 
C, we face several difficulties. First, in a CLTS the observations are related to events 
while in a POMDP they are related to states. As a consequence, we need to label 
the states by the latest observation made by the system. Secondly, our objectives are 
not based on states but on observed sequences. Fortunately, the relevant information 
pertaining to the observations, namely the information about ambiguity of observed 
sequences, is available in the belief. Thus (with two exceptions) the states are triples 
formed of a state q, an event ‘a’ and a belief B of the CLTS. A third adaptation 
concerns the control mechanism. In C, the control is performed by choosing (possibly 
randomly) a subset of allowed controllable events. Thus actions of Me are subsets of 
events that include the uncontrollable events. Given some control decision X*, to define 
the transition probability of Me from ( q , a, B) to (q 1 , a' , B'), one must consider all runs 
in C labelled by events of X* from q to q 1 such that the last event, labelled by l a h , is 
the only observable one. The probability of any such run is obtained by the product of 
the individual step probabilities. The latter are then defined by the normalization of 
weights w.r.t. X*. Finally, there cannot be infinite runs of unobservable events due to 
the convergence of C. However some runs can reach, via unobservable events, a state 
from which no event of X* is enabled. In other words, the control X* applied in (q. a, B) 
may have a positive probability to reach a deadlock {i.e. the chosen decision rule leads 
to a strategy for the CLTS which is not live). In order to capture this behaviour and to 
obtain a non defective probability distribution, we add an additional state lost, that 
corresponds to such deadlocks. The next definition formalizes our approach. 

Definition 6.9. The POMDP M<j = {Q Me , g^ c , Obs, Act, T Mc ) derived from a CLTS 
C = (Q,g 0 ,X,T) is defined by: 

• Q Mc = Q x X 0 x Blc^ {(<70, £, {go}), lost} with q™ c = {qo,£, {go}); 

• the set of observations is O = X 0 U{lost}, with Obs(lost) = lost and for (g, a, B ) 6 
Q Uc , Obs {(q,a,B)) = a; 
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• Act = {£* C E | £' D S \ S c }; 

• for every (qi,a,B) E Q Mc and S' E Act, T M e (( gi , a, 5), S’) = // E Dist(Q M ) 
where for b E X* PI X G •' 

- = 

n 

22 (n rS *^*’ a *’ %+1 )) ■ TS *0?n+i,M'); 

a l. an. b . 2—1 

<?1-r<72”*-T’Qn+l - ><? 

Cl]_ 


n 

- //(lost) = £ n tS * 9i+i); 

a l. %. 2—1 

qi >q 2 — - >Qn+ 1 

Q-i •••ct n GS*ns u 

G E (g n +i)=0 


for every S* E Act, T Mc (lost,S*) = li ost . 


Given C, the construction of Me, which is of size in can be done i 


m ex¬ 


ponential time. Also, the probability distributions over next states (/i in Definition 6.9) 


are presented as sums over runs of C, but they can be computed in polynomial time by 
matrix operations. 

A CLTS C and its associated POMDP Me are closely related. In particular, strate¬ 
gies in C and schedulers in M<; are in a one-to-one correspondence. First, let us explain 
how to naturally derive a strategy n for C from a scheduler r in M^. For an observed 
sequence a± - ■ ■ a n E £*, we set 7r(oi • • • a n ) = r(oi • • • a n ) Notice that the strategy it 
obtained that way is not necessarily live: for example, if after ai • • • o n the choice of 
r leads with positive probability to lost, then n is not live. However, as soon as r 
ensures to avoid state lost, then the corresponding strategy it is live. Similarly, to a 
live strategy n for C, we can associate a scheduler r in Me that always avoids lost: 
given a sequence of observations that does not contain lost, thus of the form a\ ■ ■ ■ a n , 
with Oj E £ 0 for all i, we set r(ai • • • a n ) = 7r(ai • • • a n ). 

Moreover, if ( 7 r, r) is a pair of live strategy and corresponding scheduler (that always 
avoids lost), the probability measures an d are essentially equivalent. More 
precisely, the product in M^ with the observation and the belief does not change the 
probability measure defined by C n . 

We now show how to decide for POMDP the existence of a finite-memory scheduler 
that ensures a Biichi objective with probability one and a safety objective with positive 
probability. We use LTL notations to denote sets of runs in a POMDP, such as O, □ 
and □<> for eventually, always and infinitely often respectively (given a state q, DO# 
thus represents the set of runs containing q infinitely often). 


Theorem 6.5. The problem whether, given a POMDP M with subsets of states F and 
I, there exists a finite-memory scheduler r such that P^([HO_F) = 1 and P^ /I (DI) > 0 is 
EXPTIME -complete. 
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Theorem 6.5 derives from Propositions |6.4| and 6.5 below, that state, respectively, 
the upper bound in the general case, and the lower bound in a particular case, namely 
for the safe active diagnosability under finite-memory strategies. 

Proposition 6.4. Given a POMDP M with subsets of states F and I, one can decide 
in EXPTIME whether there exists a finite-memory scheduler r such that P^([IIO.F) = 1 
and P^ 1 (□/) > 0. 

Due to the complexity of this proof, we decompose it using two lemmas. The idea 
is the following. We first define a set Win = i of pairs of beliefs ( B , B') with B C B' such 
that there exists a scheduler that ensures with probability 1 to stay in I from any state 
of B and to reach F from any state of B'. As B C B' , this implies that if one starts 
with a distribution which support is B' , there is a scheduler satisfying both the Buchi 
objective with probability one and the safety objective with positive probability. Such 
a belief B' is a “winning” belief. However, there are “winning” beliefs that cannot be 
obtained directly from Win=i. In Lemma 6.1 we show how to compute efficiently Win = i 


through a greatest fixed point algorithm. Using Win=i, we then build a set of beliefs 
Win, which contains intuitively the beliefs from which there exists a scheduler that can 
reach a belief that corresponds to the second component of a pair in Win=i with positive 
probability while never reaching a “losing” belief (a belief from which we cannot satisfy 


the Buchi requirement). Finally, in Lemma 6.2 we show that Win contains exactly the 
set of “winning” beliefs. Thus, there exists a scheduler satisfying the two objectives iff 
the initial belief {q^} belongs to Win. 


Proof. In this proof, the POMDP M = {Q, q$, Obs, Act, T) is fixed, and we use notation 
P^°(Ev) to denote the probability of Ev under scheduler t assuming that instead of qo, 
the initial state in M is given by the distribution ho £ Dist(Q). 

Let us first explain how to compute the following set of pairs of beliefs: 

Win=i = {{B’,B) | B' C I, B' C B s.t. 3t s.t. 

Vho with Supp(h'o) = B, P^°(nC>F) = 1 , and 
Vhp with Supp(ho) = B', p£ 0 ([H/) = 1 } . 


Intuitively, Win = i denotes pairs of beliefs such that there exists a scheduler that ensures 
a Buchi objective almost-surely from the larger belief, and a safety objective almost- 
surely from the smaller one. Note that, (1) in the definition of Win=i, we do not require 
the scheduler r to be finite-memory and ( 2 ) as schedulers associate a decision rule to 
every sequence of observation, the same choices are taken after the same sequence of 
observations for the Brichi and the safety objective although the initial distribution 
differs. Given that we consider pairs of beliefs, we introduce the following notation: 
A((H / , B), Or) = (A(H / , Or), A(H, Oi)), and similarly for sequences of actions and 
observations. Also, for X C Q & subset of states, we denote by Bl<zx = {B £ Bl \ B C 
X} the set of beliefs contained in X. 

We now show how to efficiently compute Win=i. 
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Lemma 6.1. Let Win^ be the greatest fixed point starting from {(g, B\ B) E Q x Bl x 
Bl | q E B, B' C B, B' C /} of the following operator: 


W {(g, B[,Bi) | 3n >1, 3g 0 • • • q n E Q, 3aq, ■ ■ • a n 30i • • ■ 0 n , 

(S', B 2 ) = A ((S', Si), («!, Oi) • • • K, O n )), Vg' E S 2 , (g', S', S 2 ) G W, 

5o = 5, Qn G -F, Vi < n,r(gj,a i+ i)(g i+ i) > 0,V1 < j < n, Obs(gj) = 0 J; 

Vi < n, VO', for (S', S 3 ) = A((S', S), (a,, Oi) ■ ■ ■ (a*_i, O^i)^, O')) 

we have Vg' G S 3 , (<7, S 3 , S 3 ) C W n Q x Bl<zi x £>/} . 

We have Win=i = {(S', S) | Vg G S, (g, S', S) G Wi noo }. 

Proof of Lemma \6.1\ To establish that Win = i corresponds to the projection on the pair 
of beliefs of Winoo, we first assume that for all q E S, (g, S', S) belongs to Winoo, and 
exhibit a scheduler r that witnesses (S',S) E Win =3 . Let us define r as follows. The 
scheduler r has finite memory Bl x Si. From memory state (S',S), r dictates to play 
uniformly all actions a such that for every observation 0 and every q E A(S,a,0), 
we have (g, A ((S', S), a, 0)) E Winoo- Note that this set of “safe” actions is necessarily 
non empty because (g,S',S) E Winoo- If a is played, and 0 is observed, the memory 
state of t is updated to A((S', S), a, 0), which is still in Winoo, by assumption on 
a. The scheduler r then continues similarly with memory state A((S', S), a, 0). So 
defined, let us show that r witnesses ( B', B ) E Win =3 . First, let 5q be a distribution 
with support B. The scheduler r ensures to stay (surely) in Winoo- Moreover, for every 
q E B, with a positive probability, say P^b'.b) > 0; tl ie sequence (aq, Oi) • • • (a n , O n ) of 
actions and observations leading to F that derives from the hxpoint definition, happens 
from q. There are finitely many P( q% B',B)i all are positive, so they are lower bounded 
by some positive value p. Playing r forever thus ensures visiting F almost surely, 
and iterating this reasoning, even visiting F infinitely often with probability 1. Now, 
assuming B' 0 let 6' 0 be a distribution with support B’. Any action picked by t ensures 
that, whatever the observation, the first belief-component remains in /. Therefore, 
surely, from distribution S ' 0 the plays stay in the invariant I. 

Let us now assume that the triplet (g, B ', B ) is removed during the iterative compu¬ 
tation of the fixed point W, x . We prove, by induction on k. that if (g, B', B) is removed 
at iteration k, then, (B ', B) ^ Win = i. If k = 0, the pair is removed at initialization, 
hence B 1 % I or B' % B , and obviously (B', B) (j Win = i. Otherwise it happens at the k- 
th iteration, for some k > 1. Assume, towards a contradiction, that there exists a sched¬ 
uler r, witnessing that (B ', B) E Win = i. In particular, there exists a sequence of pairs 
of actions and observations allowed by the scheduler (oq, Oi) • • • ( a n , 0 n ) so that there 
exists go - - • q n G Q with g 0 = g, q n E F, Mi < n, T(g,, a i+ i)(g i+ i) > 0, VI < j < n and 
Obs(gj) = O j. Because the triple (g, B ', B ) was removed at iteration k, it must be that, 
either (1) for (B' 2 , B 2 ) = A[(B', B), (aq, Oi) • • • (a n , 0„)), there exists g 2 E B 2 such 
that (q 2 ,B',B) ^ Wk-i, (2) no run corresponding to a sequence (aq, Oi) • • ■ (a n , O n ) 
satisfying (1) and starting in q ends in F or (3) there exists an index i and an ob¬ 
servation O' such that for (B' 3 , B 3 ) = A((-£>', B ), (aq, Oi) • • • (aj_ 1, 0,;_i)(aj, O')) there 
exists q E B 3 , (g, B' 3 ,Bz) ^ Wj-~ 1 flQx Blci X Bl. In the first case, it means that 
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there is a positive probability, under r to reach a pair of beliefs out of W^~ 1, and 
thus out of Win=i by induction hypothesis. As the sequence of action and observa¬ 
tions was chosen so that one can reach F from g, the second case implies that the 
first case holds with our selected sequence of actions and observations. For the third 
case, let (B' 3 ,Bo) = A((B', B), (cci, Oi) • • • (ag-i, Oj_i)(ag, 0()) • Either there exists 
g' £ i?3 such that (g, B' 3 ,B^) Wk- 1, then it is treated similarly to the first case. Else 
B' 3 Blci ■ Observe that, in this case, the second requirement on t is not satisfied since 
P$(□/) <1. □ 


Thanks to Lemma 6.1 Win = i can be computed in EXPTIME. Let us now define Lose 
as the set of beliefs that are clearly losing: 


Lose = {B £ Bl | -i3tV<5o with Supp(<5o) = B,F s T °(OOF) = 1} . 


As established e.g. in [BGG09] in the more general framework of 2-player stochastic 
games with signals, Lose can also be computed in EXPTIME. 

Informally, we now consider the set of beliefs from which one can reach, while staying 
in /, and not risking to fall in Lose, some belief B such that there exists B' ^ 0 with 
(B ', B) £ Win=i. In order to easily represent what staying in / means, we assume 
without loss of generality that the set of states Q \ I is absorbing Formally, let Win 
be the following set of beliefs: 


Win = {Bo £ Bl | 3(B',B) £ Win=i s.t. B’ ^ 0 and 

ztai • • • a n , 30i • • • 0„, A(L?o, («i, Oi) • • • (a n , 0 n )) = B 
Vi < n, V0-, A (Bo, (cci, Oi) • • • (a*_ i, Oi_i)(«i, 0')) ^ Lose}. 


The set Win characterizes winning beliefs, that is, beliefs from which there exists a finite- 
memory scheduler (called a winning scheduler) ensuring at the same time, the Biichi 
objective DOF almost-surely, and the safety objective □/ with positive probability. 
Formally: 

Lemma 6.2. Bq £ Win if and only if for every 5o with Supp(<5o) = Bo, there exists a 
finite-memory scheduler r such that P^(nOF) = 1 and P^°(mi) > 0. 


Proof of Lemma 6.2 Assume first that Bo £ Win. We design a finite memory scheduler 
r that is winning from any initial distribution do with support Bo. In a first mode, r 
aims at reaching a pair of beliefs (B',B) £ Win = i from Bq. More precisely, r plays 
the sequence of actions that leads with positive probability from Bq to some B £ Bl 
such that there exists B' ^ 0 with (B',B) £ Win=i. If this succeeds, r then switches 
to another mode, where it behaves as the winning scheduler that starts from (B ', B) in 
Lemma 6.1 If it fails, the play ends in a belief B\ fz Lose (by definition of Win), and 
from there r plays to ensure visiting F infinitely often with probability 1. All in all, 
r ensures almost surely visiting F infinitely often, and with positive probability (the 


4 This can be ensured similarly to what was done for the set Qf in Subsection 1.4 of Chapter [5J 


page 43 
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probability of the prefix leading to B, times the probability that the play is in B' at 
that time point) to stay in I. Note that the size of the memory t uses is in 0(\BI\ 2 ). 
Indeed, in its first phase, it tries to reach a belief by using a set of actions of length 
smaller than \Bl\ as it does not need to visit the same belief twice. If it fails to reach 
the target belief, then ensuring the Biichi requirement can be done with a belief-based 
scheduler, i.e. a scheduler that only remembers the current belief, thus with memory of 
size \Bl\. If it reaches its target however, it needs to remember pairs of beliefs as done 
in Lemma 6.1 thus requires a memory of size \Bl\ 2 . 


Let now do be an initial distribution with support Bo, and assume that there exists 
a finite-memory scheduler r such that P^°(DOi ? ) = 1 and P^°(D/) > 0. We consider 
M(t) the pLTS generated by r, with finite state space Q x Mem, where Mem is a 
finite set of memory states. Without loss of generality, we iteratively tag each state of 
M (r) with its associated belief. Since r is winning and almost surely a run reach a 
BSCC, there must exist a BSCC C in M T , reachable from some (qo, too) via an /-run 
p (a run where all state are included in I), and such that all states (q,m) G C satisfy 
q 6 I, and there exists a state ( qf,rrif ) £ C such that qf G F. Let ( q,m ) G C be the 
state reached by run p, B be the belief obtained after observing p. From ( q,m ), under 
scheduler r, all plays stay in I. Moreover, for any q' G B, from ( q',m ), under scheduler 
r, almost all runs visit F infinitely often. As a consequence, by the definition of Win = i, 
({q},B) G Win = i. Then, we conclude that Bo G Win, exploiting the /-run p, and the 
fact that r ensures DO F almost-surely, and thus always avoids Lose. Q 


Win characterizes the winning beliefs, and can be computed in EXPTIME. We thus 
showed the computability in EXPTIME of the set of supports B from which for every 
distribution <5o with Supp(do) = B there exists a finite-memory scheduler r such that 
P*>(DO F) = 1 and ?£<>(□/) > 0. P 

Now the safe active diagnosis restricted to finite-memory strategies can be reduced 
to the existence for POMDP of a finite-memory scheduler that ensures a Biichi objective 
almost surely, and a safety objective with positive probability. As M^ is exponential 
in the size of C and the algorithm on the POMDP is in EXPTIME, we obtain a 2EX- 
PTIME complexity upper-bound. Fortunately, in order to avoid a doubly exponential 
blowup and to establish the EXPTIME complexity, we observe that the exponential 
comes in both cases from the computation of beliefs depending only on the original 
CLTS. This implies that the safe active probabilistic diagnosis problem is in EXPTIME 
when restricted to finite-memory strategies. 

Corollary 6.2. The safe active diagnosis problem restricted to finite-memory strategies 
is decidable in EXPTIME. 


Proof. Given a CLTS C, we build Me and decide if there exists a scheduler r ensuring 
P^(DOF) = 1 and P£°(n/) > 0 with I = {( q,a,B) \ q G Q c } and F = {(q,a,B) \ 
B C Qf V q G Q c } and So is the Dirac distribution of support Due to the link 

between Me and C, this choice of F corresponds to runs that are either correct or surely 
faulty in C and this choice of I corresponds to runs that are correct. Thus there exists a 
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finite-memory scheduler r as defined above iff the corresponding strategy n in C ensures 
safe active diagnosis. Moreover, as explained above the corollary, deciding the existence 
of this scheduler can be done in EXPTIME. □ 


A matching lower-bound is already known from the literature: 


Proposition 6.5 ( BFHjjff]). The safe active diagnosis problem restricted to finite- 


memory strategies is EXPTIME-/iard. 


Obviously, this lower bound also holds for the more general problem: on POMDP, 
whether there exists a finite-memory strategy ensuring a Biichi objective almost-surely 
and a safety objective with positive probability. 


3 Conclusion 


Degradation of a controllable probabilistic system combines two objectives. The system 
must satisfy at the same time a diagnosability and a degradation condition. Interestingly 
and as shown first in [ BFH+14] . having to satisfy both conditions at the same time 
increases the difficulty: safe active diagnosability combines two decidable problems 
yet ends up being undecidable. In order to regain decidability, we introduced two 
new degradation notions both in a qualitative and a quantitative ways. While the 
quantitative versions are undecidable, the qualitative ones brings interesting results. 
Indeed, on the one hand, they are close to safe active diagnosability as two of the 
notions are equivalent to it for finite pLTS. On the other hand, they are decidable in 
EXPTIME. As EXPTIME is the lower bound of the complexity of active diagnosability, 
it is unsurprisingly also a lower bound of the complexity of the combination of active 
diagnosability and of a degradation condition. Therefore we can test the combination of 
active diagnosability with a degradation condition without reaching a new complexity 
class. 

This analysis however can result in diagnosers requiring infinite memory. When 
restricted to finite-memory controllers, many differences appear. First, as the pLTS 
obtained by controlling a CLTS with a finite-memory strategy is finite, then accord¬ 
ing to Theorem 6.1, safety, strong resiliency and lasting fault freeness are equivalent. 


Studying the safe active diagnosability, we showed it to be EXPTIME-complete. Thus, 
the restriction to finite memory helped regain decidability. For weakly resilient active 
diagnosability, the restriction to finite memory is not necessary as, as shown in Theo¬ 
rem [672j if the system is weakly resilient active diagnosable, there exists a strategy with 
finite memory. 

The notions of degradation introduced here were inspired from the notion of safe 
active diagnosability. One could be interested in other notions of degradations repre¬ 
senting different forms of failures within the system. For example, in our framework, 
the notion of faulty run is a Boolean one; once a fault occurred, the run is faulty. The 
fault is thus seen as a definitive and complete damage of the system. But a fault could 
only represent a small degradation of the system which would still be partially available. 
In this alternative framework, the degradation to be evaluated would be the evolution 
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of the number of faults in a run w.r.t. its length. Another possible direction of research 
would be to take a more general approach to the notion of combined objectives. It 
would be interesting to determine which pairs of objectives can be studied separately 
and which ones lead to undecidability. 



Chapter 7 

Opacity 


This thesis consists in a study of the control of the information in probabilistic systems, 
mainly focusing on diagnosis. While the goal of diagnosis is to analyse an observation 
in order to reveal a hidden information (the fault), one could be interested in asking a 
dual question: can we limit the amount of hidden information that is revealed by the 
system. This question belongs to an important domain of partial observation issues 
called opacity. While the two notions can appear similar, the motivations behind them 
are different. Diagnosability is a notion of safety of a system while opacity is one of 
security. This difference in motivations implies that the questions asked for opacity are 
not be the same as the ones asked for diagnosis. Moreover, the model itself can differ 
in order to possess different properties. In this introduction, we first present informally 
opacity and some questions related to this domain of research: in the passive framework 
then in the active one. Finally, we discuss the form of control used in the active opacity 
framework, emphasising the differences with active diagnosability. 


Opacity problems for passive systems. Given a set of secret runs, a run discloses 
the secret if every run with the same observed sequence is secret. With this definition, 
the disclosure of the secret is akin to exact diagnosability. One can also define a notion 
of disclosure that would resemble approximate diagnosability: for e > 0, a secret run 
e-discloses the secret if the probability of runs with the same observed sequence that 
are not secret conditioned on the probability of the observed sequence is at most e. For 
non-probabilistic systems, opacity boils down to detecting if there exist a run disclosing 
the secret to an observer. For probabilistic systems, we are interested in quantifying 
the opacity of the system |IBMS15 : , SH141 IBKM12] , For instance we would want to 
determine if the measure of disclosing runs of a given length is positive, if it is above a 
certain threshold, and how this measure evolves with the observed length of the runs. 
This precise quantification of the measure of disclosing runs, called disclosure, does 
not appear in the diagnosability notions we studied. Indeed, a fault was considered a 
dangerous event which cannot, in any case, be missed. For the disclosing of the secret 
however, it is more usual to tolerate that part of the secret may be leaked, as long as it 
is a limited amount. 
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Opacity problems for active systems. The focus on disclosure takes a whole new 
meaning when considering active systems. Indeed, a controller will have an effect on the 
system, and depending on its choices, this disclosure will increase or decrease. Existing 
works study the case where the controller maximises the disclosure |BCS151 IBKMSf8j . 
This corresponds to a worst-case analysis of the system, i.e. for the worst possible 
control. This kind of analysis aims at representing the case where the control is done 
by an attacker that is observing the system. This control can be obtained by the 
attacker by using a virus for example. The opposite direction, where the control tries to 
minimise the disclosure is also worthy of analysis. Indeed, for example, if a system has 
been designed in order to satisfy a specification, yet there are still some liberties within 
the system, some choices that are possible and that do not affect the specification, then 
these choices can be made in order to optimise opacity of the system. In this case, the 
control is realised during the design of the system and is made in order to minimise 
the disclosure of the system. Thus, both maximisation and minimisation corresponds 
to real issues. 


Formalisation of the control. When we studied active diagnosis in Chapter |6j the 
control was exterior to the system: from the observations it received, it was able to 
stop some controllable actions from occurring. The controller and the observer had the 
same information and thus could be thought as the same mechanism. In the examples 
given earlier (the virus for maximisation and the system design for the minimisation), 
the control comes from within the system. There is thus a clear separation between the 
controller and the observer/attacker. To formalise this opposition, the control is realised 
with a full knowledge of the system: it knows what is the exact run that is followed 
and especially what is the current state and make its decision based on this. However, 
as the attacker is not himself within the system a run only discloses the secret with its 
observation. In other words, the controller will try to minimise or maximise a set of 
runs satisfying a condition that is based on their observations. Moreover, we assume 
the attacker is aware of how the controller makes its choices. Indeed, the security of 
the system should not be based on the black box hypothesis (that an attacker is lacking 
information). Especially when considering cases such as the virus example: the virus 
could very well have been implanted in the system by the attacker, ensuring he is aware 
of how the virus works. 

In Section [lj we establish the specifications and important questions of opacity that 
we consider throughout the chapter. These definitions present two different horizons 
over which to consider opacity: a given fixed horizon and an unbounded yet finite hori¬ 
zon. In Section [2] and Section [3j we study opacity over finite horizon for maximisation 
and minimisation respectively. These two sections echo one another emphasising the 
differences between the two. Finally in Section [4] we detail our results for opacity over 
fixed horizon for both maximisation and minimisation. 


This chapter develops and extends some of the results from (BHL17a| . 
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1 Specification for Opacity 


The notion of opacity is very similar to the one of diagnosis. They both consider the 
information revealed by runs of partially observable systems. The difference is that the 
goal of diagnosis is to reveal an information about the current state of the system, while 
opacity tries to hide an information. Despite these similarities, the framework of opacity 
contains important differences with the one of diagnosis. We first define formally the 
framework of opacity for passive systems (Subsection 1.1), then extend the definitions 
to allow a control of the system (Subsection |1.2[) . 


1.1 Opacity for Markov chains 

Labelled Markov chains, as introduced in Chapter [4j are pLTS where every event is 
observable. We now define another kind of Markov chains called observable Markov 
chains which are pLTS where the observation is associated with the state instead of the 
transition. This labelling of states thus describes what an external observer can see and 
is given by an observation functio >0 We use this new framework as, while diagnosis 
aimed to detect an event (the fault), opacity consists in hiding that the system is 
currently in a secret behaviour represented by its state. This could however easily be 
translated as the detection of a transition triggered when entering a secret state. In 
fact, the equivalence of associating events with transitions or with states is a folk result. 


Figure 7.1 gives the informal idea of how to push events from states to transitions 


and Theorem |7.1| starts by a modification of the system that is close to the usual 
transformation allowing to associate the labels with the states rather than with the 
transitions. 




Figure 7.1: Pushing observations from states to transitions. 


Definition 7.1. An observable Markov chain (OMC) over alphabet is a tuple A4 = 
(S', p, 0) where S is a countable set of states, p : S —>• Dist(S) is the transition function, 
and 0 : S —>• E 0 U {e} is the observation function. 

We write p(s 7 1 s) instead of p(s)(s') to emphasise the fact that the probability of 
going to state s' is conditioned by being in state s. Given a distribution /to on S, we 

lr The equivalent of the observation function for pLTS was called mask function. 
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denote by A4(po) the Markov chain with initial distribution fig. The definitions of runs, 
observed sequences, probability measure, ... can easily be adapted to OMC. To give an 
example, an infinite run of A4(po) is a sequence of states p = So-Si • • • £ 5“ such that 
Po(so) > 0 and for each i > 0, p(sj+i|sj) > 0. The observed sequence of this infinite 
run is 0(p) = 0(so)0(si)... E ££°. The observation function is called non erasing if 
0(5) C £ (all states are visible). 

As in opacity one aims to hide a secret behaviour of the system, a way to represent 
what is secret is needed. There are various ways to define it depending on if we want 
the secret to be permanent, intermittent, described within the system.. .We consider 
here the case where the secret is permanent and given by a subset of states Sec C 5 of 
the model: a (finite of infinite) run so s i • • • is secret if Sj E Sec for some i, otherwise it 
is public. Under this choice, the secret itself behaves very similarly to the fault. As for 
pLTS where we could make the partition between faulty and correct states without loss 
of generality, we assume here that the set of secret states Sec is absorbing. To show 
this can be done without loss of generality, a new Markov chain AT = (S',p r , O') is 
defined from Af by: S' = (S X {0,1}), where (s, 0) represents state s where the secret 
has not been visited while (s, 1) represents the opposite situation. The transitions are 
then duplicated accordingly: (1) p'((s' , i)\(s, i)) = p^js) for all s E 5, s' E 5 \ Sec, and 
i E {0,1}, (2) p'((s', l)|(s,i)) = p(s'\s ) for all s E 5, s' E Sec, and i E {0,1}. The new 
observation function is defined by 0 7 ((s, i )) = 0(s) for all s E 5 and i = 0,1 and the new 
set of secrets is 5 X {1}. There is a one-to-one probability-preserving correspondence 
between the runs in A4 and the ones in AT. 


Example 7.1. Consider the OMC of Figure 7.2 with initial distribution \ qo . The 
observation associated with a state by the observation function is displayed next to it. 
The secret state is shaded. Assuming o\ ^ e and 02 7 ^ £, every state is associated with 
an observation different than e, therefore the observation function is non-erasing. 



Figure 7.2: An infinitely-branching OMC with Sec = {q s }- 


One quantitative way to define the disclosure of a system is to consider that a 
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run discloses the secret if the probability that the current run belongs to the secret, 
conditioned over the observation of the run, is greater than some given threshold e > 0. 


Definition 7.2. Given an OMC M = (S,p, 0), an initial distribution po, Sec C S and 
an observation w 6 S*, the proportion of secret runs with observation w is: 


p sec m( w )H 


^MQo)({P e 0 I P is secret}) 

PvKwoO) 


For e > 0, w is e-min-disclosing if Psec m(uo)( w ) > 1 — £ an d no prefix of w sat¬ 
isfies this inequality. Writing DF n for the set of e-min-disclosing observations, the e- 
disclosure is defined by Disc £ (At(^o)) = Y1wgd s ^he positive e-disclosure 

problem consists in deciding whether Disc £ (.M(//o)) > 0. 


To establish a parallel with diagnosability, positive e-disclosure is similar to eFF- 
diagnosability. Indeed, in eFF-diagnosability (resp. the positive e-disclosure problem) 
one considers the fault (resp. secret) revealed if the likelihood of the fault (resp. secret) 
conditioned on the observation is above 1 — e. The difference is that for opacity, we 
ask whether the measure of the set of runs disclosing the secret is positive while for 
diagnosability, we require this probability to be equal to the probability of faulty runs. In 
other words, considering a run to be faulty iff it is secret, the system is eFF-diagnosable 

iff Disc £ (AI(^o)) = T(Foc)- 

In this chapter, we aim at studying active notions of opacity. While being the most 
realistic notion of probabilistic disclosure, e-disclosure is unfortunately a too complex 
notion. Indeed, the problem is already undecidable for OMC: 


Theorem 7.1. The positive e-disclosure problem is undecidable for OMC. 


To establish this undecidability result, we reduce the emptiness problem for proba¬ 
bilistic automata. The reduction itself is pretty straightforward. Note however that the 
reduction requires to first translate the PA into an OMC such that the probability of 
acceptance of a word w in the PA is equal to the probability to end in a secret state in 
the Markov chain knowing that the run has observed sequence w. 

The emptiness problem and the value 1 problem)^] are undecidable for PA already 
with a two-letter alphabet |Paz711 IGO10| . Hence in the various reductions we use the 
alphabet {a, b}. 


Proof. Given a PA A = (Q, qo, {a, b}, (P a , P&), F) that we suppose complete without 
loss of generality, we first transform A into an incomplete OMC A where {a, b, b} is 
the observation alphabet (an illustration is given in Figure 7.3). The set of states is 
Q = Q U {(/tag | q E Q A tag E {a, &}}, with initial distribution l 9o . The observation 
function 0 is defined by 0 (q) = b and 0 (q c ) = c for q E Q and c E {a,b}. The 
transition function p is defined for q, q' E Q and c E {a, b} by p{q' \ q c ) = P c ((/, qf) 
and p(q c \ q) = \. This OMC is incomplete as for every state q E Q, the sum of the 
probabilities exiting q in A is 1/2. 


2 These notions were defined before Theorem 


4.3 
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Figure 7.3: From PA A to incomplete OMC A. 



Figure 7.4: Reduction to the positive e-disclosure problem. 


We now build the OMC JO a_= (S,p, 0) over alphabet {a, b, b, (}} by adding two 
states to complete A (see Figure 


A): 


7.4 


where the doubly circled state qj is a final state of 


S — {fipubj ^sec} U Q , with Sec — {Ssec}) 


• The function p is obtained from p by adding the transitions: For every q E F, 
P(s se c | q) = for every q E Q\F, p(s puh \ q) = and p(s pub \ s pub ) = p(s sec \ 

Ssec) — 1) 

• 0 extends O by 0(s sec ) = 0(s pub ) = (t- 


We now prove that, given e e]0, 1 [, A accepts a word with probability strictly greater 
than 1 — e iff Disc £ (A4(/Uo)) > 0. First assume that there exists a word w = a b ... a n G 
{a, b}* with P J 4 (u>) > 1 — e. Then w corresponds to a non secret run with observed 
sequence w = \>a\\>.. .a n b in JO a and Psec j v f ( w )(u)0) = > 1 — e, which implies 

Disc e (A4(/io)) > 0. 
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Conversely, if Disc 6 (At (/io)) > 0, then there exists an observation w' in ({a, b, b, fl})* 
such that Psec_yvi( /i0 )(rt; / ) > 1 — e. In this case, w' is of the form barb ... a n bjttt* where, let¬ 
ting w = a±... a n , we have Psec m(h 0 )( w ') = Psec_ M(/i0 )(baib ... a n bj)) = P a(w). There¬ 
fore £>i_ e (A) is not empty. □ 

This undecidability result leads us to consider the simpler case where the disclosure is 
the probability of the set of runs surely leaking the secret, i.e., such that all runs with 
the same observation are secret. One such disclosure notion, the cu-disclosure (used 
in [BCS15[ 1BMS151 IBKMS16] ). was defined for a Markov chain At = (S,p, 0) with 
initial distribution po by considering a measurable set of secret runs SecRuns C 
In our context, as mentioned earlier, SecRuns is Reach(Sec), the set of infinite runs 
visiting a state from Sec. Moreover an infinite observation w G discloses the secret 
if all runs p G 0^ 1 (w) are secret. Setting SecRuns = flM(uo) \ SecRuns, we define: 

Definition 7.3. For an OMC At = (S,p, 0), an initial distribution po and a subset 
Sec C S, with SecRuns = Reach(Sec), the cu-disclosure of Sec in At is: 

Disco,(At (po)) = (SecRuns \ 0~ 1 (0(SecRuns))). 

The downside of this definition is that it only considers infinite observed sequences. 
In reality, an attacker will only have access to finite observed sequences before having 
to deduce if the system is in a secret state. To obtain measures directly related to the 
finite observation of a potential attacker, we assume that At = (S,p, 0) is convergent'. 
each infinite run p has an infinite observation 0 (p) G TF. Two measures can then be 
defined: using fixed or finite horizon. In the fixed-horizon case, the attacker observes 
the system for a fixed amount of time and has to make his deduction at the end of 
this observation. In order to link the amount of time the attacker observes the system 
and the number of observations they receive, in this case, we only consider non-erasing 
observation functions 0. In the finite-horizon case, the attacker can wait as long as 
they want, as long as it is a finite amount of time. 

Definition 7.4. Let Af = (S,p, 0) be an OMC, po an initial distribution and Sec C S. 
A finite observation w G X* discloses the secret if all runs p G 0 _1 (in) are secret. It is 
min-disclosing if it discloses the secret and no strict prefix of w does. 

n-disclosure : When 0 is non-erasing, we denote by D n , for n G N, the set of disclos¬ 
ing observations of length n. The n-disclosure (disclosure with fixed horizon n) is 
Disc n (M{po)) = J2wGD n ¥ M(no)( W )i 

Disclosure : Writing D m [ n for the set of min -disclosing observations, the disclosure 
(w.r.t. finite horizon) is defined by Disc(At(//o)) = X^eAnm 

Note that if D is the set of disclosing observations, and V(po) = U w& d U pg o-i(w) 
Cyl(p) the set of runs disclosing the secret, then Disc (At (/io)) equals F M (fj, 0 )(V{po))- 
As waiting longer gives more information, the disclosure with finite horizon is always 
at least as large as the disclosure with fixed horizon. In fact, (Disc n (At(/io)))ngN i s a 
non-decreasing sequence with limit Disc(Af (po)). 
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Example 7.2. Consider the infinitely-branching OMC of Figure 7.2. The single secret 


run is SecRuns = o- n d its observation is of. Moreover, the observations of the 

public (as opposed to secret) infinite runs is O(SecRuns) = o\of. As a consequence, the 
infinite observed sequence of the secret run discloses the secret, Disc^, = ^. However, 
no finite observation is disclosing: Vn G N, Disc n = Disc = 0. 

This shows that disclosiLre and u-disclosure differ. 


However, both notions coincide for convergent finitely-branching OMC. 

Lemma 7.1. Let At = (S,p,0) be a Markov chain, po an initial distribution and 
Sec C S. For SecRuns = Reach(Sec), Disc(.A/f (po)) < Disc w (Af (po)) with equality when 
At is convergent and finitely branching. 


Proof. We first establish the following claim: if At is convergent and finitely branching, 
then the set of runs p such that 0 (p) has length n is finite for any n > 0. 

We first prove the claim for signalling runs. We proceed by induction on the ob¬ 
servable length. There exist finitely many signalling runs of length 0: by convention 
they are the runs that (1) do not contain any event and (2) start in an unobserv¬ 
able state of Supp(po) 0 0 _1 (e). Let us assume the hypothesis holds for n G N. 
For every signalling run po with |po| 0 = n we consider the tree formed by the set 
O n+ \ = {p G SR j po P p A 10 (/?) | = n + 1} by sharing common prefixes. Internal nodes 
of this tree correspond to unobservable states while all leaves are observable. Since the 
OMC is finitely branching, the tree is of bounded degree. By contradiction, assume that 
the tree is infinite. Konig’s lemma yields an infinite branch containing only unobservable 
states, which contradicts the convergence hypothesis. Therefore there exist only finitely 
many signalling runs of observable length n + 1 extending po- As there exist finitely 
many signalling runs of observable length n according to the induction hypothesis, one 
deduces that there exist finitely many signalling runs of observable length n + l|^ This 
concludes the induction. The result can then be extended to every runs as, from the 
convergence hypothesis, for every n G N and every run p of observable length n, there 
exists a signalling run p' G SR„_|_i such that p P p'. As |SR„+i| < oo, there are finitely 
many runs of observable length n. 

We now prove that the set of infinite runs V = U me n U pg o-i(«,) Cyl(p) is contained 
in SecRuns\0~ 1 (0(SecRuns)). Let p\ be an infinite run in V. Then there is a disclosing 
observation w\ G E* and a signalling prefix p\ of p\ such that 0(p / 1 ) = w\ and p\ is 
secret. For any infinite run p 2 such that O(pi) = 0(p2), the observation w\ is also a 
prefix of 0 (p 2 ), hence there is a finite signalling prefix p' 2 of p 2 such that O(p'fi) = w\. 
Since w\ is disclosing, p' 2 is also secret, hence p\ belongs to SecRuns\ 0 _1 (0(SecRuns)) 
and Disc(Ad(po)) < Disc w (Ad(/ro))- 

For the converse inclusion, let p be an infinite run in SecRuns\0 -1 (0(SecRuns)) with 
observation 0 (p) = w = 0102 ■ ■ ■ G E'A We prove by contradiction that there is a finite 
disclosing prefix w of w and a signalling prefix p of p such that p G Cyl(p) and 0(p) = w. 

3 For the case n = 0 one must also consider the signalling runs obtained from runs starting in states 
of Supp(po) \ 0 _1 (e). There are finitely many such runs too. 
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Otherwise, for any n > 1, w n = o\.. .o n is not disclosing and there exists a signalling 
run p n such that 0 (p n ) = w n but p n is not secret. The set T = {p' E SR | 3n p' < p n } 
of all signalling prefixes of the p n ’s form a tree: the root of the tree is £ and the nodes 
at level k are the prefixes with observation Wk, {p' E T | |0(//)| = k}. A node p" is a 
child of p' if |0(p / )| = Wk, |0(p ,, )| = w^+i for some k and p' A p". From the claim, we 
know that T is of bounded degree. Assuming that it is infinite, Konig’s lemma again 
yields an infinite branch p^ such that each prefix of length k is not secret and has 
observation w^. Hence p 0 0 is not secret and has observation 0(p oo ) = w, which is a 
contradiction. □ 

In the following we only consider finitely-branching convergent systems. As a con¬ 
sequence, we will only focus on disclosure over fixed or finite horizon. 


1.2 Opacity for Markov Decision Processes 

We now want to add control to the system. The form of control we define here differs 
from the one used in Chapter [6] Indeed, in the context of diagnosability, the controller 
observed the system and from this observation, they chose some controllable actions 
that they blocked until the next observation. Thus the control had to make decisions 
without an exact knowledge of the state of the system. For opacity we are interested 
by a control acting with full knowledge. Therefore, the control can be more accurate. 
We use Markov decision processes where in each state there is a set of possible actions 
and the controller chooses one of them. This action induces a probability distribution 
on the next state reached. 


Definition 7.5. An observable Markov Decision Process (OMDP) over alphabet E is 
a tuple M = (S, Act,p, O) where S is a finite set of states, Act = U<; e sM(s) where A(s) 
is a finite non-empty set of actions for each state s E S, p : S x Act —> Dist(S') is a 
(partial) transition function defined for ( s,a ) when a E A(s) and 0 : S —> E U {e} is 
the observation function. 


The difference with POMDP described in Definition 6.8 


page 


6.8 beside some 


syntactic modifications is on how the control is defined. As for OMC, we write p(V|s, a) 
instead of p(s,a)(s'). We use the same kind of definitions as usual for runs, observed 
sequences, ... For example, given an initial distribution po> an infinite run of M is 
a sequence p = So«oSi fl i • • • where po(so) > 0 and p(sj_|_i|sj, af) > 0, for s t E S, 
ai E A(sj), for all i > 0. We denote by M(po) the OMDP M with initial distribution 
po- F° r decidability and complexity results, we assume that all probabilities occurring 
in the model (transition probabilities and initial distribution) are rational. 


Example 7.3. Consider the OMDP of Figure ]? .5\ From the initial state qo, two actions 
are possible. If action a is chosen, the system moves to q\ with probability 1/2 and to 
q 2 with probability 1/2. If b is chosen, every state has a probability 1/3 to be reached. 


The OMDP model uses both non-deterministic choice (the choice of the action) 
and probabilistic choice (the induced distribution). The non-determinism is where the 
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o 2 

a, 1 


a, 1 
oi 

Figure 7.5: An example of OMDP where the initial distribution is a Dirac distribution 
on (/o- Transitions are labelled by a set of pairs of actions and of the probability to take 
this transition under that action. 


control can be operated. It is resolved by a strategy which associates with every run 
a distribution on the actions enabled at the last state of the run. Given a finite run 
p with last(p) = s. a decision rule of an OMDP for p is a distribution 5 £ Dist(A(s)) 
representing the action chosen after p. For such a decision rule 5, we write p(V|s, 5) = 
EaeA(s) S(a)p(s'\s,a). 

Definition 7.6. A strategy for the OMDP M = (S', Act ,p, 0) is a mapping a associating 
to every finite run p a decision rule cr(p). 

Given a strategy a, a run p = soao s i«i • • • of M is er-compatible if for all i, a* £ 
Supp((7(soa 0 siai ... s*)). 

In order to apply the strategies as defined here one requires to remember the whole 
run that occurred. Moreover, the strategies are allowed to choose randomly between 
the different allowed actions. All of this may not always be necessary however. We 
are thus interested specifically in strategies satisfying specific properties. A strategy 
cr is deterministic if er(p) is a Dirac distribution for each finite run p. In this case, 
we denote by er(p) the single action a £ A(last(p)) such that cr(p) = l a . A strategy 
ex is observation-based if for any finite run p, cr(p) only depends on (1) the observed 
sequence 0(p) and (2) the current state last(p), i.e. given p' such that 0 (p) = O(p') 
and last(p) = last(//)> we have a(p) = cr(p'). We then write cr(0(/?), last(p)) for u(p). 

Let (7 be a strategy and p be a u-compatible run. We define Bf the belief of p w.r.t. 
<7 about states as follows: 

= {s £ S | 3p er-compatible, 0 (p) = O (p) A s = last(//) A 0(s) e}. 

The belief Bp contains the set of states that can be reached under the strategy cr and 
with observation 0(p). A strategy cr is belief-based if for all finite run p, cr(p) only 
depends on the belief Bp and the current state last(p), i.e. given p' such that Bp = B°, 
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and last(p) = last(p / ) , we have <j(p ) = u(p'). Observe that a belief-based strategy is 
observation-based since B° only depends on w = O(p). So we also write B ^ for Bp. A 
strategy o is memoryless if a (p) only depends on last(p) for all p. 

The semantics of a OMDP M with initial distribution po under the strategy a is a 
(possibly infinite)observable Markov chain M CT (/xo) where each state is associated with 
a finite a-compatible run of M(//o), that can be equipped with the observation function 
mapping 0(last(p)) to the state associated with the finite run p. The transition function 
p a is defined for p a finite run and p' = pas' by p a {p'\p) = cr(p)(ajpCs'ls, a) and we denote 
by P Mct(mo ) (or P(j for short when there is no ambiguity) the associated probability 
measure. Writing V a {p o) for the set of runs disclosing the secret in M a (po), we have 
Disc(M cr (/xo)) = PM CT ( Ato )(V 0 -( / uo)). We assume all OMDP considered are convergent 
(there is no cycle of unobservable states), which implies the convergence of all OMC 
induced by strategies. 


Example 7.4. Consider the OMC of Figure | 7. 6| It represents the semantics of the 


OMDP of Figure 7.5 with the strategy cr choosing the action b initially, then always 
choosing action a. a is observation-based as the only run for which it does not select a 
is the empty run, which is the only run with observed sequence 02 . It is also belief-based 
as the empty run is the only run with belief {qo}- Indeed, after some observations, the 
current belief is either {qo,q 2 } or {gi}. It is however not memoryless as the empty run 
and qobqo both ends in qo but the same action is not chosen in both cases. After three 
observations, under a, the system cannot be in qo any more, it is thus necessarily in a 
secret state. Therefore, Disc(Mo-(/io)) = 1. 


02 °2 



Figure 7.6: The OMC induced by the strategy choosing the action b initially, then 


always choosing action a for the OMDP of Figure 7.5 


The control can be either adversarial or cooperative with respect to the system: it 
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can try to either maximise or minimise the opacity. We therefore define the disclosure 
value of an OMDP according to the type of the strategies. We only consider e-disclosure 


for fixed horizon in light of the undecidability result of Theorem 7.1 


Definition 7.7. Given an OMDP M = (S, Act,p, 0), an initial distribution po and a 
secret Sec C S, for disc £ {Disc, Disc n , Disc^j}, n £ N and 0 < e < 1, the maximal dis¬ 
closure of Sec in M is disc max (M(/ro)) = sup^ disc(Mo-(/ro)) and the minimal disclosure 
of Sec is disc m i n (M(/xo)) = info-disc(M CT (/xo)). 

Note that the construction ensuring that once a secret state is visited, the run 
remains in a secret state forever, extends naturally from OMC to OMDP. We only 
consider OMDP of this form in the rest of this chapter. 

Example 7.5. Consider the OMDP of Figure \7.fy As soon as the strategy selects action 
a, the system enters a secret state and discloses the secret with probability 1. Therefore 
Disc max (M(^o)) = 1- If the strategy only selects action b however, observing a ‘o\’ 
clearly shows the system is in q\, thus disclosing the secret, while after observing at 
least two ‘ 0'2 the belief is {go, Q 2 } which does not disclose the secret. The probability to 
observe at some point ‘o\ ’ being equal to 1/2, Disc m i n (M(^o)) = 1- 

We study the following problems for OMDP over finite or fixed horizon: 

• Computation problems. 

— The value problem: compute the disclosure; 

— The strategy problem: compute an optimal strategy whenever it exists. 

• Quantitative decision problems. Let 1 x 1 = > for maximisation and 00 = < for 
minimisation. 

— The disclosure problem: Given M and a threshold 9 £ [0,1], decide if disc(M) 00 
0 ; 

— The strategy decision problem: decide if there exist a strategy a such that 
disc(Mo-) 00 6. 

• Qualitative decision problems. 

— The limit-sure disclosure problem: the disclosure problem when 9=1 for 
maximisation and 9 = 0 for minimisation; 

— The almost-sure disclosure problem: the strategy decision problem when 9 = 

1 for maximisation and 9 = 0 for minimisation. 

For the complexity results regarding a fixed horizon n, we assume that n is written 
in unary representation or bounded by a polynomial in the size of the model where 
the polynomial is independent of the model as done in classical studies (see for in¬ 
stance |PT87] 1. 
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As said earlier, the whole power of the strategies we defined may not be necessary to 
answer the above problems. Restricting ourself to a subset of strategies that gives the 
same disclosure values can help simplify the proofs and the representation in practice 
of these strategies. Moreover, it helps understanding what is important in the control 
of an OMDP to optimise the disclosure. We thus show that for disclosure problems we 
can restrict strategies to observation-based ones. 

Proposition 7.1. Given an OMDP, a secret and a strategy o, there exists an observation- 
based strategy o' such that for disc G {Disc, Disc n , Disc^}, disc(Mo-(/io)) = disc(M CT /(/xo))- 

For this proof, from an arbitrary strategy o, we build an observation-based strategy 
o' with the same disclosure value. The strategy o' is randomised and is obtained by 
choosing, after an observed sequence w, a distribution on the different choices made by 
o on runs with observed sequence w. This is done so that the probability of choosing 
an action after observing w is the same for both strategies. 

We then prove that o' meets the same disclosure value as o. More precisely, we 
establish that the probability to reach a state with a given observation is the same for 
both strategies. This is done by induction on the length of the observed sequence and 
on an ordering of the unobservable states. Two cases have to be considered, depending 
on if the last state is observable or not. However, each case is dealt with in the same 
way (both for the initialisation and for the induction step). Thus we only detail the 
first one. 


Proof. Let M = (S', Act, p, 0) be an OMDP with initial distribution po, and let o be a 
strategy. For an observation w G £* and a state s G S, we define the sets (note that 
these are finite sets given the claim in Lemma 7.1) R(w, s ) = {p finite run of M CT (po) | 
0 (p) = w A last(p) = s}. 

We now define a mapping o from E* x S’ to Dist(Act) by 


o(w , s) 


1 

^cr(p) 


p£R(w,s) 


d(w, s ) corresponds to the average choice made by o after a run with observed sequence 
w and ending in s. Using o, we define the new strategy o' for a finite run p by 
o'(p ) = o(0(p), last(p)). We claim that P a i(R(w, s )) = Pcr(i?(rc, s)) for any observation 
w and any state s, which entails equality of disclosure. 

Partitioning the set of states into S = S a 1+) S u where S u = 0^ 1 (e), we can assume 
a topological sort on the subgraph obtained by removing all edges in S X S 0 (this 
subgraph is acyclic due to the hypothesis of convergence). This means that there exists 
a numbering r/ of the states so that if rj(s') > p(s), there is no transition from s to s'. 
We proceed to prove the above claim by a joint induction on the pairs (w, s ) using |u;| 
and r](s). 

For the base cases, we need to establish the property for w = e with s G S u , and for 
i«GS with s G S 0 , where /jlq(s) > 0 in both cases. 
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Case 1. By induction on rj(s), we consider a state s £ S u such that rj(s) = mim,/^ 

Then P CT /(i?(e, s)) = po (s) = P CT (i?(£, s)). Assuming the property holds for (e, s) with 
rj(s) < n, we prove it for s' with r/(V) = n + 1. We have: 


P (T /(i?(£,S / )) = Po(s') + E E P( S, ! S ’ a ) E F v'(P) a '(P)( a ) 

s£S u ,p(s)<p(s') agA(s) p€R(e,s) 


and using the definition of a' yields: 


¥ a r(R(£,s')) = p 0 (s') + E Y |s,a)d(e,s)(a) E P <t'0) 

s£Su,v(s)<ri(s') aeA(s) p£R(e,s) 


= P 0 {s')+ Y Y P( s '\ s ’ a 

s£S u ,v(s)<p(s') agA(s) 


E P gfl( £ , 3 ) ^ApMp)(a) 

YlpeR(e,s) ^cr(p) 


p£R(s,s) 


Applying the induction hypothesis on (e,s) yields Yl P eR(e s) p o-'(p) = P<j'(^( e ) s )) = 
P a{R(e,s)) = EpeR( £ , s ) P a(p) thus: 


P 0 -/(i?(e,s / )) = Po{s')+ Y Y p (’ s ' I s ’ a ) E F <r(p) a (p)( a ) = ^a{ R (s,s')). 

s£S u Pl(s)<ii(s') a&A(s) p£R(e,s) 

Case 2. We now consider w = o £ H and s' £ S Q , hence 0(s / ) = o. Then: 

P a /(i?(o,s')) = Po(s') + Y Y P ( S '\ S ’ a ) E ¥ <r'(p) a> (p)( a ) 

sGS u aG.4(s) p£R(e,s) 

and a reasoning similar as above yields the result. 

For the induction step, we first need to prove the property for ( w,s') with s' £ S u . 
assuming it holds for all (w, s ) with s £ S Q and for all (w, s ) with s £ S u and rj(s) < ij(-s'). 
Then we have: 

F a ’(R{w,s')) = Y Y P( s '\ s ’ a ^ Y ¥ <T'(p) a '(p)( a ) 

sSS 0 a£A(s) p£R(w,s) 

s£Su,v(s)<ri(s') 

and we can conclude along the same lines as above. 

Finally, we consider (w' , s') with w' = wo £ £*£ 0 and s £ S Q , with: 

P a '(R(w',s')) = Y Y P( S '\ S ’ a ) E ¥ <r'(P) a '(P)( a ) 

s£S aEA(s) pER(w,s) 


which again implies the desired result. 


□ 
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As seen in the previous proof, erasing observations leads to technical and cum¬ 
bersome developments. In order to avoid them in the design of procedures for the 
finite-horizon case, we apply the preliminary transformation that ensures the observa¬ 
tion is non-erasing described in the next proposition. We precisely state the size of the 
obtained OMDP in view of complexity results. 

Proposition 7.2. Given an OMDP M = (S, Act,p, 0), an initial distribution /jq and a 
secret Sec, one can build in exponential time an OMDP M' = (S', Act ,p', O'), an initial 
distribution p' 0 and a secret Sec 7 where 0' is non-erasing and for disc G {Disc m j n , Disc maa; } 
disc(M(^o)) = disc(M / (/iQ)). In addition, the size of S', p' and p' 0 is polynomial w.r.t. 
the ones of S, p and //q. The size of Act' is polynomial w.r.t. the size of Act and 
exponential w.r.t. the size of S. 

The main idea of the construction is that every time a run visits an observable 
state, an observation-based strategy can fix a set of action for the current state and 
for every unobservable state. It will then keep this choice until the run visits a new 
observable state. Once such a set of actions is fixed, one can easily compute the proba¬ 
bility distribution to reach the next observable state. Unobservable states can thus be 
removed from the system. We also add a new state to deal with the possibility that an 
unobservable state had a positive probability in the initial distribution. 

Proof. We first build the new OMDP and then explain the correspondence between 
strategies in both models, which induces the relationship between disclosures. 
Construction of the OMDP. We start from OMDP M = (£, Act,p, 0) with Act = 
U se sA(s), observation alphabet S, and a set of secret states Sec C S. Choosing a fresh 
observation symbol [j and a fresh state s#, we build an OMDP M' = (S', Act', p',0') 
with set of states S' = {sj} U (S \ 0~ 1 (e)), and observation alphabet X U {)}}, where 
the initial distribution is 1 S(J . The observation function 0 ; is defined by O^sjj) = [j and 
0'(s) = 0 (s) otherwise. Note that all states have non-trivial observation. The set of 
actions of M' is Act 7 = DR where DR is the set of vectors of deterministic decision rules 
5 over S, i.e. such that 5(s ) G A(s). The intuition of DR is that the actions associated 
with the current state and any unobservable state by the strategy after an observation 
is fixed until the next observable state, so we can gather this set of action into a single 
action. 

We now define the transition probabilities, starting by the transitions exiting sj. 
For a run p = SQai...a n s n , we write n(p) = Y\ff = i p(si|sj_i, af) and first(p) = so- 
Given an observable state s G S and 5 G DR, the set E(s$, 5, s ) contains the finite runs 
p = -SoaiSi.. .a n s n starting from some so £ Supp(^o) and ending in s n = s such that 
ai = d(sj_i) for all i, 1 < i < n and all states So, • • •, s n -i are unobservable. Observe 
that the intermediary states are all distinct due to the convergence of the OMDP. We set 
p'(s | sjt,<S) = E pe E(s h ls) ^(first(/o))7r(p). If s G Supp(^ 0 ), the set E(s t ,5,s) contains 
the run reduced to p = s. 

We turn to the transitions exiting the other states. It is easier as we do not need 
to take the initial distribution into account. Given a state s ^ Sjj, an action 5 G DR 
and an observable state s', we consider the finite set E(s,5,s') of signalling runs of 
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M p = sqcl l • • • CLnSn starting in s® = s and ending in s n = s' such that for each 
i, 1 < i < n, ai = 5(si- 1 ), and all intermediate states are unobservable. We set 
5) = J2 p ^e(s 5 s') Note that E(s,S,s') may include runs like p = s 5(s)s'. 

In order to efficiently compute the transition function of some 6, one uses a topo¬ 
logical sort of the unobservable states thanks to the convergence hypothesis, and then 
compute the probability from observable states to reach first the unobservable states 
topologically sorted and then the observable states. This gives a polynomial time com¬ 
putation of the transition function of <5. Thus, the size of S' , p' and p' 0 is polynomial 
w.r.t. the ones of S, p and pq. Moreover, the size of Act' is polynomial w.r.t. the size 
of Act and exponential w.r.t. the size of S. 


Correspondence between strategies. The above construction ensures that any run 
p' = ... 5kSk of M' corresponds to the set of runs p = pisi,... pkSk of M containing 

the sequence si... s*, of observable states, with p\ G E(s$, S\, si) and pi G E{si- 1 , 6i, s*) 
for 1 < i < k. All runs in the set have the same observation w = O(si). .. 0 (sk) with 


O'(p') = $w. 

To show that disclosure over finite horizon is the same in both OMDP, we establish 
correspondences between the strategies of M and IVT and the associated disclosure value. 
From Proposition 7.1 we can restrict to observation-based strategies. 

• Let a' be an observation-based strategy of IVK, defined on j)S* X S'. Given an obser¬ 
vation w G £* there exists 6 such that for every state s G S', we have cr'($w, s ) = 5. We 
define a(w,s) = 5(s). Then, writing P CT (resp. P CT /) instead of Pm ct (/x 0 ) (resp. Pm',(^)), 

and defining for w G E* and s G S \ 0' 1 (e), R(w,s ) = {p G SR m AmO | Q(p) = 
w A last(p) = s} and R'(w,s) = {p' G | O(p') = jju; A last( / o / ) = s}, we have 

P a(R(w, s)) = P a'{R'{w, s)). 

• Conversely, given an observation-based strategy cr of M, we build an observation- 
based strategy a' of M' as follows: Given wGP, we define the mapping a'($w) : S —> 
Dist(DR) by <7 , (j|u;)(s) = a(w,s) for any s G S. Then, using the same notations as 
above, we have P a (R(w, s)) = P a >(R'(w, s )). 

Therefore, defining the set of secret states of S' by Sec' = Sec n S', as the set of 
secret states is absorbing, the disclosures over finite horizon are equal for a and a'. □ 


Thus for finite horizon, one can restrict oneself to non-erasing observation functions 
with some care on the complexity of the actions. Also, on fixed horizon, we assume 
the observation function is non-erasing. Thus in both cases we are able to use this 
assumption. 


2 Maximisation with finite horizon 


We start the study of the disclosure problem with the maximisation objective over finite 
horizon. In other words, here the strategy tries to maximise the disclosure of the secret 
after an arbitrarily long, yet finite, amount of time. 

we show how to restrict the study to deterministic strategies 


In Subsection 2.1 


without loss of generalities. We show that most of the notions are unfortunately un- 
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decidable in Subsection |2.2 
Subsection 


and prove the decidability of the almost-sure disclosure in 


2.1 Deterministic strategies are sufficient 

We showed in Proposition |7.1| that one can limit oneself to observation-based strategies. 
In fact, for maximisation problems, one can do even better. Indeed, the additional 
power given by randomisation is not useful and thus observation-based deterministic 
strategies are sufficient. 

Proposition 7.3. Given an OMDP M, a secret Sec and a disclosure notion disc G 
{Disc, Disc n , Disc^j, for any observation-based strategy o there exists a deterministic 
observation-based strategy o' such that disc(M CT (/io)) < disc(M cr /(/ro)). 

This proof strongly uses Lemma 1 of |CDGH10| (or alternatively |GS14| ) which 
establishes, in an active stochastic setting, that deterministic strategies are sufficient to 
optimise an objective defined by a set of infinite runs. This Lemma does not directly 
give the result we want as, contrary to the objectives used in their paper, the choice 
of the strategy modifies which runs are disclosing. However, as a disclosing run for a 
randomised strategy is also a disclosing run for a deterministic strategy that does not 
introduce new runs, we can use parts of their proof to show our result. 


Proof. In the proof of Lemma 1 of |CDGH10j . the authors show that a randomised 
observation-based strategy can be seen as a convex combination of a family of deter¬ 
ministic observation-based strategy. As a consequence, in our framework, given an 
observation-based strategy a and a disclosure notion disc, there exists an observation- 
based deterministic strategy oset such that for every finite run p, Supp(<7w P f(p)) C 
Supp(cr(p)) and )(HrOo)) > Pm ct ( mo )0MW)))- 

The second property is not enough to conclude, as a disclosing run under o is not 
necessarily a disclosing run under o'. However, thanks to the first property we can 
obtain that V a (po) n fl^detb 10 ) c V adet (po)- Indeed, as a is more permissive than Od e .t , 
^ M <r det (ho) C HM (J (/xo). This implies that, given a run p, if 0 (p) discloses the secret 
with the strategy a then either 0 (p) discloses the secret with the strategy o,iet or 0 (p) 
cannot be observed with a^et- 

This implies: P CTdet (V CT (/x 0 )) = P CTdet (V CT ( M o) G H M ^(«)) < P CTdet(w) (V ffdet M). 

Therefore, disc(M ffdet 0zo)) = P ff<fct (V ff<Jet (|i 0 )) > ^a det (V a (p 0 )) > P< T (V <7 (/i 0 )) and 
the result holds since P (T (V (T (^o)) = disc(Mo-(^o))- □ 


Observe that this proof shows that the restriction to deterministic strategy does not 
decrease the disclosure. However, it does not necessarily keep the same disclosure as 
before contrary to the proof used to restrict to observation-based strategies. Therefore 
it cannot be applied for minimisation problems. 
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2.2 Undecidability of the disclosure and limit-sure disclosure prob¬ 
lems 


As mentioned in the previous proof, one of the difficulties of opacity is that the set of 
disclosing runs depends on the strategy: a transition can be completely blocked by some 
strategy, modifying the set of disclosing observations. This was illustrated in Figure [A5| 
where choosing action a in state go removes the edge to go- This situation was excluded 
in the computation of the disclosure presented in |BKMS161 IBKMS18] where the authors 
study a restricted form of Interval Markov Chains |JL91j . The disclosure problem for 
the general class of OMDP was left open. We answer negatively to the general problem 
by proving undecidability of the disclosure problem, hence the disclosure cannot be 
computed in general. Undecidability also holds for limit-sure disclosure. 

Writing 1 for the set of intervals in [0,1], an interval Markov chain (IMC) over 
an alphabet E is a tuple M = (S. Si ni t-I, 0) where S is the set of states, Si n u is the 
initial state, / : S —> I 5 associates with every state s G S' a mapping from S to I, and 
0 : S —> £ U {e} is the observation function. We abuse notations by writing /i £ /(s) 
to denote any distribution p : S —> [0,1] such that for all s' G S, p(s') G I(s' \ s). 
The notion of run p is the same as for an OMC but a transition from s = last(p) to 
some successor requires the choice of a distribution p G I(s). A strategy of IMC M 
is thus a mapping a associating with each finite run p with s = last(p) a distribution 
cr(p) G I(s). In other words, an IMC is a OMDP where the chosen action represents a 
set of probabilities satisfying the interval conditions set by I and summing to 1. In fact, 
an IMC can be transformed into an (exponentially larger) OMDP where actions are the 
basic feasible solutions of the linear program specified by the constraints associated with 
intervals |SVA06l ('SI 108 . Thus undecidability results for IMC also hold for OMDP. 


Example 7.6. Consider the IMC of Figure 1.1 From the initial state, the strategy 
must attach a probability p± G [0, |] to the transition to s i and a probability p 2 G [|, 1] 
to the transition to S 2 ■ As, in order to obtain a distribution, we require that P 1 +P 2 = 1? 
P 2 is de facto restricted to the interval [^, 1]. If p\ = \ is selected, then P 2 = \ and the 
run has a probability p\ to move to si and P 2 to move to S 2 ■ In these two states, the 
only exiting transition is labelled by the interval {1} which we simplified by removing 
the braces in the figure. Thus, the run then loops indefinitely on the state. 
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Figure 7.7: Example of IMC. 


Theorem 7.2. The maximal finite-horizon disclosure problem is undecidable for OMDP, 
even when the secret is reached with probability 1 and for a non-erasing observation func¬ 
tion. 
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The maximal finite-horizon disclosure problem when restricted to finite-memory strate¬ 
gies is undecidable (even with the same additional assumptions). 

Starting from a PA A, we build an IMC M ^ = (S, so, I, 0) such that there exists 
a word w G {a, b}* with P^(w) > } if and only if Disc max (M J 4 ) > The proof of 
Theorem |7.2| is more involved than the proof of Theorem |7.1| because the strategies 
must be taken into account. The goal is to have the strategy choose a single word then 
"plays" it in the IMC and the probability to disclose the secret is half of the probability 
of the selected word. However, just with this, nothing would prevent the strategy to 
switch words during the run if it realises that the current run will not disclose the secret. 
We add a second component that ensures that if the strategy deviated from the single 
selected word the current run will not disclose the secret. Doing so, the strategy loses 
the advantage of knowing the current state of the run. 


Proof. We first give the construction in two steps. The first step is very similar to what 
was done in the proof of Theorem 7.1 Starting from a PA A = (Q,qo,{a,b},T, F) 


that is supposed complete, we build an IMC A = (Q,qo,I,0) where £ = {a, b} is the 
observation alphabet. The set of states is Q = Q U {q c \ q G Q A c G {a, 6}}, with initial 
state qo- The observation function 0 is defined by 0 (q) = £ and 0 (q c ) = c for q G Q 
and c G {a, b}. The interval mapping I : Q —> IN is defined for q,q' G Q and c G {a, b} 
by: 

• Hq' I Qc) = T(q' | q,c) is a point interval; 

• Hqc I q) = [o, i]. 

Compared to the illustration given in Figure [A3| this construction amounts to replacing 
all b by e (making the states non observable) and the probabilities \ from original states 
to new ones by the interval [ 0 , 1 ]. 

However, the construction of the complete IMC M 4 = (S, sq, 1,0) from A is more 
involved and requires to add a supplementary gadget limiting the power of the strategy. 
This is why we first use an observation function which can erase states and explain at 


the end how to relax this hypothesis. The construction is illustrated in Figure 7.8 with 


some conventions to avoid too many edges, a final state from A ( e.g. like qf) is doubly 
circled. 


• S = {so,s 1 ,ql,q^,q\ ) ,q s } U Q U {s c | c G {a, b, #}} U {r c \ c G {a,M,b}}; 

• J(si I so) = I(qo | so) = I and the restriction of / to Q is I. For all c G {1, a, b, jj}, 

c' G {a,b, tt}, I(s c i | s c ) = g and I(r c t \ s c ) = [0, |], for all c, d G {a, 6 , tf, b}, 

I{r c j | r c ) = \ and I(q s \ r c ) = g. For all q G Q \ F, I(qj \ q) = [0,1], for all 

Q G F, I(q% I q) = [0,1], and I(q s \ qj) = I(q b | qj) = I(q s \ qf) = I(q s \ q s ) = 1. 

• 0 extends 6 by: O(s 0 ) = O(si) = e, O(qj) = 0(g # 2 ) = 0(q s ) = fl, 0 (qf = b, for 
all c G {a, 6, fj, b}, O (r c ) = c, and for all c G {a, 6, jj}, 0(s c ) = c. 




214 


Opacity 


1 1 
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Q.Q 



Figure 7.8: Reduction from the strict emptiness problem of PA to the maximal hnite- 
horizon disclosure problem. An edge outgoing from a dotted box should be duplicated 
to originate from all states in the box and an edge entering a dotted box from a state 
s should be duplicated from s to any state in the box. Hence a loop on a dotted box 
represents a complete graph inside the box (including self-loops). 


Informally, for Sec = {q s }, the upper gadget ensures that for any strategy a there 
is at most one word w € {a, 6 }* such that the observation w\ Jb(j discloses the secret. 
The lower gadget allows one to generate secret runs of observation w \jbjj with half the 
probability as the one assigned by the PA to w. 

We now formally prove that there exists a word w G {a, b}* with (w) > | if and 
only if Disc max (M t \) > 

First suppose there exists a word w = a\...a n G {a, b}* accepted with probability 
greater than | in A. We define the strategy cr for a finite run p in both parts of 
(when relevant) as follows: 

• In the upper part, assume that p ends in a state s c with c G {1, a, 6,(1}. If there 
exists i < n such that 0 (p) = a\... a>i then <r(p)(r ai+1 ) = 0, leaving no choice for 
the rest of the distribution: In order for the sum of probabilities to be equal to 1 
we have for b 7 ^ aj+i, a(p)(rb ) = If 0 (p) = w, then <T(p)(rj) = 0, which also 
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leaves no choice for the rest of the distribution. 


• In the bottom part, we can assume that p ends in a state q E Q. If there exists 
i < n such that 0 (p) = a\ ... a,i then a(p)(q ai+1 ) = 1- Finally, if O (p) = w then 
a(p)(q^) = 1 if q E F, and a(p)(qj) = 1 otherwise. 

At the beginning the system will move with probability \ to A, where the strategy 
ensures that the word rcff is observed. This leads to the state qj with probability ^P^(to) 
and thus the next observations belong to bjf* and the runs with observations in rc)jt>tt + 
belong to the secret. On the other hand, the system can also go to si with probability 
2 from where, due to the decisions of the strategy, a run with observation rcff ends in sj 
(the decisions of the strategy ensure that either the run does not have observation w ft, 
or it could not go in a r state). Moreover, from s*, b cannot be observed. This implies 
that wffbff is a min-disclosing observation in M^, hence Disc(M_ 4 !fr ) > }P(( (w) > 
Since DisC m a Y (M 4) = sup^ Disc(M J 4 jf7 ), we can conclude that DisC m a Y (M 4) > \- 

Conversely, suppose that the disclosure is strictly greater than | and let o' be a 
strategy such that Disc(Mn !fr ) > Then, a must forbid states in {r c | c E {a, b, JJ}}, 
otherwise there would be no disclosing observation since every observation can be sim¬ 
ulated once a state r c is reached. Writing S = {a, b, jj}, we inductively define the word 
U X by a sequence (wi)i> o of non-decreasing prefixes of w: 

• We start with wq = e; 

• Assume wf is built and let pi be a run ending in state s x for some x E {1, a, b, ((}, 
with 0 (pi) = wl. If cr(pi)(r c ) = 0 for some c E {a, b, $}, then Wi+i = wfc , otherwise 
wi+i = w~i- 

The set of ambiguous observations (be. corresponding to both secret and non-secret 
runs) are the ones reaching the set of states {r c \ c E {a, b, (J, b}}: 

U W iX (ZU{\> })*f- 

vJiX / W -(T 
i^b 


Hence, the set of disclosing observations is reduced to either w tjbfj^, where w is the largest 
prefix of w in {o, b}* if ft occurs in w. and empty otherwise. Since the disclosure is greater 
than 0, we obtain w J|b(j as the single min-disclosing observation with Disc(M_ 4 i(T ) = 
Pm a CT (u;ttb(t). Since P^(tc) > 2. Pm a CT (wttb(j), we can conclude that F^(w) > 

The proof can be extended with a non-erasing observation function by replacing 
e with a fresh symbol (like in the proof of Theorem 7.1). This requires to slightly 
modify the parts of the IMC corresponding to the sets of states {s c | c E {o, b, ft}} and 
{r c | c E {a, b, ff}} in order to ensure alternation of letters from {o, b} and this new 
symbol. 

The undecidability result holds even when restricted to finite-memory strategy as 
the strategy defined in the first direction of the proof only uses finite memory. □ 
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If we could compute the maximal fiuite-horizou disclosure, we could solve the asso¬ 
ciated decision problem. Thus, as a consequence of the previous theorem, we obtain: 

Corollary 7.1. The maximal finite-horizon disclosure of an OMDP cannot he computed. 

We now turn to the qualitative disclosure problems, and using a reduction from the 
value 1 problem in PA, we also have: 

Theorem 7.3. The maximal finite-horizon limit-siLre disclosure problem is undecidable 
for OMDP. 


Proof. The reduction from the value 1 problem for PA done here is similar to the one 
of the proof of Theorem 7.2 The difference is that any run initially moving from sq 


to Si (thus moving to the part of the IMC which was used to limit the power of the 
strategy) will now almost-surely disclose the secret. More precisely, the construction of 
depicted in Figure [7)8] for the proof of Theorem 7.2 is slightly modified as follows 
(see Figure 7.9): a new state q§ with 0 {qf) = tj is added in the upper part just before 
reaching the secret state q s . In this case, the runs reaching the secret in the upper part 
disclose the secret as they end with tjJfA 

The disclosure on the bottom part is performed as before. As a consequence, if 
a word w is "selected" by the strategy, the finite-horizon disclosure will be equal to 
1/2.(1 + P^(w)). This value can be arbitrarily close to 1 iff A accepts words with 
probabilities arbitrarily close to 1 , which yields the result. □ 



Figure 7.9: Modification of Figure |7.8| for limit disclosure. 


2.3 Decidability of the almost-sure disclosure problem 

Fortunately the maximal finite-horizon almost-sure disclosure problem is decidable. The 
proof relies on results for partially observable MDP (POMDP) described in Defini¬ 
tion [678j page 

Theorem 7.4. The maximal finite-horizon almost-siLre disclosiLre problem in OMDP is 
EXPTIME-comp/ete. Moreover, if the system is almost-surely disclosing, one can build 
a belief-based strategy with disclosure 1. 

We reduce the almost-sure disclosure problem for maximisation in OMDP to almost- 
sure reachability in POMDP. The POMDP we build is exponential in the size of the 
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original OMDP and the algorithm to solve almost-sure reachability is exponential in the 
size of the POMDP |CDH10| . A naive application of the two successive results would 
give a 2-EXPTIME algorithm. However, a finer analysis yields EXPTIME complexity of 
our algorithm as these two exponentials do not stack. The hardness is obtained by a 
reduction from the safety problem in games with imperfect information that was shown 
to be EXPTIM E-complete in [BD08] , The reduction for the lower bound is similar to 
the one of Theorem 17.21 


Proof. We start by giving the construction of the POMDP, then we show that solving 
almost-sure reachability in the POMDP is equivalent to the finite-horizon almost-sure 
disclosure problem for maximisation. Finally we prove the hardness. 

Construction of the POMDP. We start from an OMDP M = (S, Act,p, 0) with 
Act = U se 5 A(s), observation alphabet X 0 , and a set of secret states Sec C S. Thanks 
to Proposition |7.2[ we can assume 0 to be non-erasing (the potentially exponential blow 
up of the number of actions does not affect the complexity result as we will see later). 
Let no be an initial distribution. We assume w.l.o.g. that po is a Dirac distribution 
on some state so £ S. We build a POMDP M' = (Q, qo, Obs, Act, T) with set of states 
Q = Sx 2 s , with qo = (so, {so}) and observation alphabet X 0 . The observation function 
Obs is defined by Obs((s,H)) = 0(s). The set of actions of M ; is Act' = DR where DR 
is the set of vectors of deterministic decision rules 5 over S. Given a state ( s,B ) £ S', 
an action 6 £ DR and an observable state s', we have 


T((s,B),5)(s',B') 


p(s'\s,6(s)), for B' = {s" | 0 (s") = 0(s')A 
< 3s £ B,p(s"\s, 6(s)) > 0} 

0 , otherwise. 


Correspondence between strategies. To show that M is almost-surely disclosing 
for Sec iff Sec X 2 Sec can almost-surely be reached in Mb we establish a correspondence 
between the strategies of M and the scheduler of Mb From Proposition 7.3 


we can 


restrict to deterministic observation-based strategies for M, and from |CDH10j . we also 
restrict to deterministic schedulers for Mb 

In both direction of the equivalence, given a strategy a and a scheduler r, we write 
P CT (resp. P T ) instead of Pm ct (^ 0 ) ( res P- P|W(r))> and define, for w £ X* and s £ S, 
the sets of finite runs R(w, s ) = {p finite run of M CT (//o) | O(p) = w A last(p) = s} and 
R'(w, s ) = {p 1 finite run of M'(r) | O(p') = w A last(//) = (s, B£f)}. 

• Let t be a scheduler of Mb defined on £*. We define o for any observation w £ X* 
and state s £ S by a(w,s) = t(w)(s, B*). Note that a r-compatible run p' of M' 
ends in a state ( s,B ) where B = (the belief w.r.t. o) if O(p') = w. We have 
P a {R(w, S)) = P T (R!{W, s)). 

Now let Reach (Sec x 2 Sec ) be the set of runs reaching Sec x 2 Sec in M'(r). Then 
we claim that Disc max (M (T (/xo)) = P r (Reach(Sec x 2 Sec )). Indeed, an observation w 
discloses the secret under strategy o iff all observable states reachable with observed 
sequence w belong to the secret, i.e. iff Bfj C Sec. Thus the runs p' in M' with a 
disclosing observation for M are the ones for which last(p / ) £ Sec X 2 Sec . Therefore, 
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thanks to the earlier remark, we have that the probability of reaching Sec x 2 Scc in M 7 
under strategy r is also the probability of disclosing Sec in M under strategy a. 

• Conversely, given an observation-based strategy a of M, we build a scheduler r 
of M 7 as follows: we define the mapping t(w) : X* —> Dist(Act) by t(w)(s, Bffi) = 
(j(w, s ) for any s £ S. With the same reasoning as above, we immediately get that 
the probability of reaching Sec X 2 Sec in M 7 under strategy r is also the probability of 
disclosing Sec in M under strategy a. 

We can conclude that M is almost-surely disclosing if and only if the runs of M 7 
reach almost-surely the set Sec x 2 Scc . Moreover, if M 7 almost-surely reaches the set 
Sec x 2 Sec , we can build a scheduler r doing so. Using the transformation described 
above and the results from jCDHIOj . we extract from t a belief-based strategy cr of M 
that almost-surely discloses the secret. 

Let us argue that the whole algorithm is in EXPTIME. The exponential in the algo¬ 
rithm of jCDHIOj comes from a determinisation of the system, which is already done in 
our transformation from M to M 7 , and thus not required a second time. Moreover, the 
non-erasing assumption on the observation could have created exponentially many new 
actions, which are exactly the ones built by the use of a vector of decision rules in our 
construction. Thus the exponentials do not stack. 


The hardness is shown with a reduction from safety games with imperfect informa¬ 
tion. 


Definition 7.8. A safety game with imperfect information is defined by a tuple Q = 
(L, £q, X, A, O, F, obs) where 

• L is a finite set of locations with initial location £q E L; 

• X is a finite alphabet; 

• A C L x Ti x L is the transition relation such that for all f £ f and a E X there 
exists at least one with (£,a,F) E A; 

• O is a finite set of observations, and F C O are the final observations; 

• obs : L —>• O is the observation mapping. 


A safety game with imperfect information Q is a turn-based game played by two 
players Control and Environment. It starts in location £q with Control to play. In the 
first round, Control chooses a letter a q E £, then Environment chooses a location i\ 
such that (£o,ao,£i) E A and Control only observes oi = obs(I'i). The next rounds 
are played similarly and Control wins if for all i, Oi fL F. The problem of existence 
of a winning strategy for Control is EXPTIME-complete |BD08j . We now describe a 
reduction from this problem to the almost-sure disclosure problem of OMDP. 

The reduction is similar to the one in the proof of Theorem 7.2 except that we 
replace the probabilistic automaton by a safety game Q = (L, £q, E, A, O, F, obs) with 
imperfect information and directly build an OMDP M = (S, Act, p, O) over alphabet 
(O U {tt, b, t]}) U E X (O U {jj, b}), with: 
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• S = { sq , £ 0 , , g 2 , q b , ql , <£ } U {4 | t G L,c G ExO}U{s c | c G E x (OU{jJ})}U{r c 

c G E x (O U {ft, b})}; 


• Act = E; 

• For all a G E,o G 0,t c G S/ G obs _1 (o), p{£' aj0 I F c , a) > 0 iff (£,a,£') G A. If 
£ G obs _1 (.F) then jj( c/i | £ c , a) > 0 and if £ £ obsthen | t c ,a) > 0. For 
all a G E, c G {0} U(Ex(OU {#})), ( b ', o')GSx(OU {ft}), p{s(bp 0 ') I «c, a) > 0 
and if b' ^ a, p{r^ b i 0 i\ \ s c , a) > 0. For all c/ GEx(OU {((}), p( r c' I r cj a) > 0 
and p(g 2 | r c , a) > 0. For all a, a' G E, p(q] \ qj,a) = p{% | q 2 , a) = p(q] \ q\ n a) = 
P(<il I q\,a) =p{q 2 s | q 2 s ,a ) = 1 . 


• O(so) = O(£o) = obs(4); For z G L,s,r, a G S, o G O, 0(^ a;O ) = (a, o); For 
o G {)j, b}, 0(2 a , o ) = o, and O(^) = O(q^) = jj = O(ql), 0 (q b ) = b, 0(g 2 ) = t]. 

The initial distribution is po(so) = 1/2 = po(£o) and the secret is Sec = {ql,q 2 }. 


This proof being similar to the one of Theorem 7.2 we only detail here the differ¬ 
ences. A run starting in so will almost surely trigger a \ and disclose the secret. A run 
starting in £o will almost surely reach ql as after any action in the copy of Q there is a 
positive probability to reach qj or c/ 2 . In order for a finite run starting in £q to disclose 
the secret, it cannot go through qj and should not share its observed sequence with a 
run ending in a state r c . Given a strategy a of M, if there exists a cr-compatible run 
p visiting a state £ c with an observation 0(£ c ) G E x F, then there is a cr-compatible 
path p' visiting ql. therefore a set of runs with positive probability do not visit the 
secret. Thus a deterministic strategy almost surely disclosing the secret in M never 
visits a state triggering an observation of the form ExF. Moreover such a strategy 
does not take the current state into account. Indeed, let p and p' be two runs such that 
0 (p) = 0 (p') and ending in two states £ c and s c . If cr(p) = a and cr{p') = a' are two 
actions in E with a ^ a! then there exists o G O such that pa£ afi is a cr-compatible 
run. Since a ^ a ', p'ar a)0 is also a cr-compatible run with same observation than pa£ a ,o- 
Hence no observation prefixed by 0 (pa£ a)0 ) would disclose the secret. 

Therefore, similarly as in Theorem 7.2, Control has a winning strategy iff there 


exists a deterministic strategy considering only the observed sequence that almost-surely 
discloses the secret. This implies EXPTIME-hardness of the maximal finite-horizon 
almost-sure disclosure problem. □ 


In this section on the maximisation of the disclosure over finite horizon, we saw 
that although we could restrict ourselves to deterministic strategies, most of the finite- 
horizon problems are very complicated (all but one are undecidable). 


3 Minimisation with finite horizon 

We now turn to minimisation over finite horizon where strategies try to hide the secret 
from an observer, and thus to minimise the disclosure. 
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This section shows a surprising result. Indeed, in Subsection 3.1 we show that 
for minimisation we cannot assume strategies to be deterministic any more, thus the 
problem seems more complex. However, the disclosure value can be computed as we 
establish in Subsection 13.21 


3.1 Deterministic strategies are not enough 

After the proof of Proposition |7.3[ we remarked that the proof showed that deterministic 
strategies do not decrease the disclosure, thus the proof could only work for maximi¬ 
sation. In fact, the result itself is limited to maximisation as randomisation may be 
necessary to minimise the disclosure. Let us see that on an example. Consider the 
OMDP depicted in Figure 7.10 with Sec = {q 2 , ^ 3 }- There are two deterministic strate¬ 
gies, choosing respectively o or b in qo. In both cases, the disclosure is l ,. On the other 
hand, for randomized strategies cr p such that cr p (qo) = pa + (1 — p)b with 0 < p < 1 , 
there are no disclosing observations, hence the disclosure is 0 . 


<?3 Qi 



Figure 7.10: With Sec = {q 2 ,q 3 }, deterministic strategies are not sufficient for minimi¬ 
sation. 


While we cannot restrict ourselves to deterministic strategies, we still use in the 
decision procedures a restricted class of strategies. These strategies are called families 
of almost-deterministic strategies and are based on e -decision rules. 

Definition 7.9. Let 5 be the deterministic decision ride for state s selecting action 
a £ A(s). Then, for e > 0, the e-decision rule d £ £ Dist(A(s)) is defined by: 

1. If |A(s)| > 1 then d £ (a) = 1 — e and for all b £ A(s) \ {a}, d £ (b) = r^4prj-/ 

2. Else 5 e = l a . 

5 £ is said to favour a. 

e-decision rules are used to define approximations of deterministic strategies. 

Definition 7.10. Let a be an observation-based deterministic strategy. Then {cr £ } £> o 
is a family of observation-based almost-deterministic strategies defined for any state s 
and w £ T, n by: a £ (w, s) = cr(w, s) 2 -n £ . 
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In other words, given a strategy cr we define a family of strategies that have an 
increasingly high probability to play like cr as the run goes on, yet always allow the other 
actions with positive probability. We will see that strategies of this form are dominant 
in the sense that they are sufficient to compute the disclosure value. However, if there 
exists a strategy that minimises the disclosure, this strategy does not always belong to 
a family of almost-deterministic strategy. 


3.2 The minimal disclosure value is computable 


Using Proposition |7.2[ we assume in the following that the observation function we 
consider is non-erasing. The complexity of the transformation does not affect the re¬ 
sults since the polynomial complexity in the number of actions is dominated by the 
exponential complexity in the number of states. 

In order to compute the minimal disclosure value, we build from an OMDP M, 
another OMDP M m i n which is a “correct abstraction” (as is stated by Proposition 7.4) 


for reducing minimal disclosure problems to minimal reachability problems. Intuitively, 
in M m j n , the states are enlarged by the maximal belief that can occur independently of 
the action that has been selected. 

Given a set of potential current states B and a new observation o, we define the 
maximal set of potential next states NextMax(P>, o) over decision rules applied to B by: 


NextMax(H,o) = {s' 6 0 1 (o) | 3s E B 3a E A(s) p(s'\s,a) > 0}. 


NextMax is intuitively the belief obtained with a strategy allocating some probabilities 
to every action. Observe that given a family of almost deterministic strategies {cr £ } and 
a run pas of M with 0(s) = o, one has for all e > 0, Bf^ s = NextMax(Hp e , 6). Then 
M m j n is formally defined as follows: 

• <Smin> the set of states, is defined by: 5’ m j n = {(s,B) \ s E B C 0 _1 (0(s))}; 

• for every (s, B) E S m in , A(s,B ) = A(s); 

• for every (s, B), ( s ', B') E S min , 


p((s',B')\(s,B),a) 


if B l = NextMax(P>, 0(s')), 
0 otherwise; 


• for every (s, B) E S min , 0(s,B) = 0(s). 

Given po an initial distribution over S, the associated initial distribution p m i n over 5 m i n 
is defined by /x m in(s, Supp(/ro) O 0 _1 (0(s))) = po(s) and /J m in(s, B) = 0 for all other B. 
We define the subset Avoid(Sec) C S min by Avoid(Sec) = {(s,B)\B C Sec}. 

Proposition 7.4. The minimal disclosure value for Sec in M(po) is equal to the minimal 
probability to reach Avoid (Sec) in M m i n (/j m i n ). Furthermore it is asymptotically reached 
by a family of belief-based almost deterministic strategies. 
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We show this result in two steps. First we show that, using families of belief-based 
almost-deterministic strategies, one can obtain a disclosure value in M arbitrarily close 
to the minimal reachability probability in M m ; n . This ensures that the disclosure value 
is below this probability. Second, we show that the disclosure obtained by an arbitrary 
strategy is greater or equal to the probability to reach Avoid(Sec). 


Proof. We know that the minimal reachability probability for Avoid(Sec) in M m ; n (^ m i n ) 
is obtained by a memoryless deterministic strategy er m i n that selects some action a Sj £ 
in state ( s,B ) (see e.g. |IBK08l| ). Consider {cr e } the family of belief-based almost- 
deterministic strategies defined by favouring a S) B hi state s after a run p such that 
Bf e = B. Given a run p = sqOq ... a n -is n in M(/io) we inductively define the run 
b(p) = (s 0 ,S 0 )a 0 . ..a n -i(s n ,S n ) in M min (/i min ) by: S 0 = Supp(/r 0 ) n O _1 (O(s 0 )) and 
Si -)-i = NextMax(Si, 0(sj+i)). Due to the observation given when introducing NextMax, 
with strategy cr £ , the observation of run p discloses the secret iff b(p) reaches Avoid(Sec). 
Consider, under strategy a e , the probability to disclose the secret with runs p such that 
b(p ) includes at least once an action not selected by cr m i n . By construction, at each step 
i. the probability of not choosing the action favoured by a e is A., hence the probability 
of this set of runs is £T >0 (1 — e)*^ < 2e. Consider now a finite run soao... a n _ i s n 
such that b(p) is n m i n -compatible. Then the probability of the original run is less than 
or equal to the probability of its corresponding run. So we deduce that the minimal 
disclosure value of M(/io) is bounded above by u + 2e where v is the minimal reachability 
probability for Avoid(Sec) in M m j n (/i, m i n ). Since this holds for all e > 0, we obtain that 
the minimal disclosure value of M(//o) is bounded above by the minimal reachability 
probability for Avoid (Sec) in M m i n (/r m i n ). 

Conversely consider an arbitrary strategy a in M(jUo)- This strategy may also be applied 
in M min (/j min ) by forgetting the second component of the state, defining a strategy 
a'. For any run soao •.. s n in M„(jUo), there is a single run (sq, So)o,q ... (s n , S n ) in 
M m i n (/r m i n ) under a' with the same probability. Given the run so°o • ■ • s n , consider 
the successive associated subsets of beliefs according to a, Bq, ..., B n . By induction 
(and definition of M m ; n ) it is straightforward to show that Bi C Si. So s^ao ... s n does 
not disclose the secret in M under a implies that (sq, So) ... (s n , S n ) does not reach 
Avoid(Sec). This entails that the reachability probability of Avoid(Sec) in M m i n (/i m j n ) 
under a' is less than or equal to the disclosure probability in M(/xq) under a. C 


Since minimal reachability probability in OMDP can be computed in polynomial 
time(see e.g. |BK08] )^| we immediately obtain the first part of the next theorem. We 
establish the second part (PSPACE-hardness) in the proof of Theorem 7.8 as the proof 
holds also for disclosure over fixed-horizon. 


Theorem 7.5. The minimal disclosiLre value of M(^o) can be computed in EXPTIME. 
The associated decision problem is PSPACE -hard. 

4 Note that since observations are not useful for this reachability objective, observations could be re¬ 
moved from the OMDP M m i n , yielding a Markov decision process, which is the model studied in IBK08I . 
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We now turn to the existence of a strategy that achieves the minimal value. As 
remarked earlier, we cannot assume such a strategy belongs to an almost-deterministic 
family of strategies. We establish that it can be analysed without additional complexity. 

Theorem 7.6. The existence of a strategy that achieves the minimal disclosure value 
can be decided in EXPTIME. In the positive case, this strategy can be computed in EX- 

PTIME. 

The main ingredient of the proof is an elimination algorithm that removes iteratively 
the beliefs from which no strategy can reach the maximal disclosure. Once al these 
beliefs were removed, if the initial belief was not deleted, then there exists a belief- 
based strategy minimising the disclosure and this strategy plays in order to stay within 
the beliefs kept by the algorithm. 

Proof. Let us first introduce multiple notations that are used within the proof. We define 
disc*(M(s, B)) as the minimal disclosure value when starting in M in state s with belief 
B. Given some belief B and some decision rule vector 5 over B we introduce the possible 
successors of B when applying 5: Next (B.S) = IV I 3s £ B 3a £ Supp(<5[sl) pfs'ls, a) > 
0} and Next(S, <5, o) = Next (B, 5) n O^(o). 

The algorithm simultaneously solves the existence and the synthesis problem. 

First, using Proposition [7Tj the algorithm computes for all (s, B) £ S^nim disc*(M(s, B)). 
Then it maintains a set Win of beliefs initially set to all beliefs from which it iteratively 
eliminates items and stops when no more elimination is possible. Given B £ Win, it 
looks for a decision rule vector 5 over B such that: 

• for all o £ 0(Next (B,§)), Next(.B, <5, o) E Win; 

• for all s £ B, disc*(M(s, B)) = E E <f[s])disc*(M(s / , Next(.B, 5, o))). 

oes s'eO-ho) 

If such a 5 does not exist then B is eliminated from Win. In other words, a belief 
is eliminated if there does not exist a decision rule that meets the minimal disclosure 
value. Each iteration can be performed in polynomial time w.r.t. |S , m in| and the number 
of iterations is at most | SAm |. Observe that when a belief is eliminated, it should not be 
“reached” by a strategy that obtains the minimal disclosure value. So the elimination 
is sound. 

When the elimination stops, the algorithm answers positively iff for all o £ 0(Supp(^o)) ; 
Supp(^o) O O _1 (o) E Win. Thus, by the soundness of the elimination step, if the answer 
is negative there is no optimal strategy for minimal disclosure value. 

If the answer is positive, let us consider the belief-based strategy a defined by applying 
the decision rules obtained during the last iteration of the algorithm. On the one hand, 
under a when visiting a state s with belief B such that disc*(M(s, B)) = 0, one never 
leaves such kind of pairs of states and beliefs. So the secret is never disclosed, showing 
that the disclosure value obtained by a for such ( s,B ) is null. Under a. the disclosure 
value of all the other pairs of states and beliefs fulfil the equations of the elimination step. 
It is known that the single solution of this system is the vector of minimal reachability 
probabilities of Avoid in M m i n (// m i n ) (see |BK08| for instance) which yields the result. □ 
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4 Fixed-horizon problems 


We focus now on fixed-horizon problems for both maximal (Subsection |4.1[ ) and minimal 
(Subsection 4.2) disclosure. In both cases, the algorithms and hardness results have 
similarities. 


4.1 Maximal disclosure 

In order to compute the value of the maximal disclosure within a fixed horizon, one 
could build the POMDP described in the proof of Theorem |7.4| then use pre-existing 
results on POMDPs. This would result in an EXPTIME algorithm, whereas we provide 
below an algorithm with a better complexity in PS PACE. 

Theorem 7.7. The fixed-horizon maximal value (when the horizon n is described in 
unary representation) is computable in PSPACE and the fixed-horizon maximal disclo- 
siLre problem is PSPACE -complete. 

Due to the complexity of the proof, we separate the algorithm computing the value 
from the hardness of the decision problem in two separate proofs. 

In order to compute the value, we first order the observation alphabet X. Then, a 
non-deterministic decision procedure operating in PSPACE enumerates every observed 
sequence of length n while maintaining the sets of states that were possible after every 
prefix of this observation, the actions that were chosen non-deterministically in these 
states and values used in the computation of the disclosure. The information kept is of 
polynomial size and when every observation has been read, one of the values computed 
are exactly the disclosure of the system at time n. This provides an N PSPACE algorithm 
which can be turned in to a PSPACE one using Savitch’s Theorem [Sav70] . In order to 
get the value we observe that we can compute the polynomially sized denominator of 
this value and then we make iterative calls to the decision algorithm. 

Proof. We first present a non-deterministic procedure that decides in NPSPACE the 
disclosure problem. It can then be determinised using Savitch’s Theorem |Sav70| . 

From an arbitrarily ordered observation alphabet X, the procedure operates as fol¬ 
lows for horizon n: 

• It maintains a disclosure value v, a sequence of observations oq ■ ■ ■ Oi with i < n, 
a sequence of sets of states B\ • • • B{ with Bj C 0 _1 (oj) for all j < i 1 an action 
aj tS E 4(s) for all (j, s ) with j < i and s E Sj, and for all (j, s ) with j < i and 
s E Bj the probability pj s to reach s after the sequence of observations oq ■ ■ ■ op, 

• Initially v = 0, oo is the smallest observation in 0(Supp(^o)), where po is the 
initial distribution, Bq = Supp(jUo) Cl O _1 (oo) and po,s = Ao(s) f° r s E By, 

• If i < n then for all s E Bi, the procedure guesses an action a^s E T(s). Let 
oy |_i be the smallest observation such that there exists a state s E B{ and a state 
s' E 0 -1 (oj + i) with p(s'|s,aj iS ) > 0. Then Bi + \ is set to {s' E 0~ 1 (oj + i) | 3s E 
Bi p(s'\s,a itS ) > 0} and for all s' E B i+1 , p i+1>s > = J2 s eB t Pi,M s '\ s ^ A, s ); 
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• If i = n, the procedure examines B n . If B n C Sec then v = v + ^2 s£ B n Pn,s 
otherwise v is unchanged. Afterwards it “backtracks” to the greatest 0 < i < n 
such that there exists o\ > Oj with some s E -Bj_i and a state s' E 0 _1 (o() 
with p(s , |s, ai-\ )S ) > 0. Then B{ and the Pi s >’s are updated accordingly and the 
procedure carries on. If there is no such i, the procedure returns to i = 0 and 
similarly looks for some o' Q > oo, where the initialisation step is again performed 
except for the value of v which is unchanged. When the maximal observation 
in X n 0(Supp(/xo)) is handled, the procedure terminates by comparing v to the 
threshold. 

The correctness of the procedure follows from the fact that there exists an optimal 
deterministic strategy where the selection of the action for the current state depends 
only on the sequence of observations (and not on the sequence of visited states). 

The space complexity of the procedure is in 0(?r|5|(log(|A|) + nb )) where b is the 
maximal number of bits used to represent a transition probability of the OMDP. 

Observe now that, since the maximal value is obtained by a deterministic strategy, 
one knows a denominator of this value: it is d n where d is the least common multiple of 
the denominators of the probabilities occurring in the model. Its bit size is polynomial 
w.r.t. the size of the model. So by iteratively solving the disclosure problem for -A for 
increasing values of i, one computes the maximal value in PSPACE. □ 

As can be seen in the proof, the optimal strategy could be computed when solving 
the value problem. However the size of this strategy may be exponential due to the 
beliefs and thus this strategy is computable in EXPTIME. 

For the hardness result, we reduce the validity of Quantified Boolean Formulae 
(QBF). Recall that QBF extends propositional formulas by allowing quantification over 
the Boolean variables. Syntactically, the formulae are described by the following gram¬ 
mar: 


4> ::= V 7 I | Vx.cj) 

::= x | if} A ip | ijj V ip \ —'ll: | true 

A QBF is closed if every Boolean variable is bound by a quantifier. Deciding if a 
closed QBF is valid (z.e. equivalent to true) is PSPACE-hard |Sip06| . 

The idea of the reduction is the following. Given < f> a closed QBF (w.l.o.g. in 3CNF 
with n variables and m clauses), we build an OMDP M such that cj) is t rue iff tl ie 
disclosure of M is greater or equal to ^ in 2(n + m) + 3 steps. In fact, is exactly 
the measure of runs reaching the secret in 2(n + m ) + 3 steps, thus every path reaching 
the secret must be disclosing. Such a run discloses the secret iff some Boolean variable 
of (j> and its negation {x and —>x for example) do not occur in its observation. 

In M, during the first 2 n steps, an assignment is ‘given’ to each Boolean variable: 
(i) for each existentially quantified Boolean variable x, the strategy chooses whether x 
or —ix occurs in the observation and (ii) for each universally quantified Boolean variable 
y, by a random choice with probability During the last 2 m steps, the strategy must 
trigger a Boolean variable in every clause of (j) so that if a clause is not satisfied by the 
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Figure 7.11: Reduction of the validity problem to the disclosure on a fixed horizon. The 
box S Xl is represented in Figure [7. 12 


current assignment, then a Boolean variable is observed as both true and false during 
the run. Thus the observation would not disclose the secret. 


Proof. We reduce the validity of a quantified Boolean formula: Given a closed QBF in 
3CNF (j> = 3x{s/yi3x2 ■ ■ ■ Vy n ?/b with if = /\?:=i m( z h V %i 2 V Zj 3 ), we build an OMDP M 
such that (j) is true if and only if Disc 2 ( n+m ) + 3 )max (M) > , 2 l n -. 


The OMDP M = (S, Act,p, O) (depicted in Figure 7.11 with some conventions to 
avoid having too many edges) is defined by: 


• S = {s in it,s end ,s^}U{s Zi | z e {x,y},i = 1.. .n}U{s^ 2i | z E {x, y}, i = 1.. .ra}U 
{st | * e {1, • • • ,m}} U {s Zi i | i G {1,.. .,m},j € {1,2,3}} U ie{1 ,... n} (S Xi U S Vi U 
S-, x . U S^ yi ) where for Zi one of the Boolean variable, S Zi = {sa’ 1 \ a £ {(t, Zi, ~>Zi \ 
z € {x, y}, i e {1,..., n}}} U {s ?’ 2 | a e {j), b, z u —>Zi \ z € {x, y}, i e {1,..., n}}}. 
Similar for the box S-, 2i of the negation of a variable. 

• A(smit) = {xi,~>xi}, and for all i < n, A(s yi ) = A(s^ Vi ) = {x i+1 ,-^x i+1 }. For 
i € {1,..., m}, A(s\) = {1, 2, 3} and M(s) = {next} for all other states (even in 
boxes). 
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Figure 7.12: Representation of the box S Xl . 


We use the convention of Figure 7.8 


• p(s a I Si n u, a) = p(sa' 1 I s ini t,a ) = 1/2. For all i < n , p{s a \ s Vi ,a ) = p(sa’ 1 \ 

s Vi , o) = p(s a | s^ yi ,a) = p(sa l | s ^ Vi , o) = 1/2. For all z < n, £>(% | 
s Xi ,next ) = I s Xi ,next) = p{s^ Vi \ s Xi ,next) = p^s^ 1 | s Xi ,next) = 

p(s yi | ~<s Xi ,next) = pisH’ 1 \ s^ Xi ,next) = p{s^ Vi \ s^ Xi ,next) = p^s^ 1 \ 
s^ Xi ,next) = 1/4, and p(sj \ s$,next) = 1. For all i = 1... m,j £ {1, 2, 3},/j(.s >z *j | 
s\,j) = 1, and if i < m, p{s l t +1 | s z ^',next) = 1. Finally, p(s en d | s Zm ?,wexf) = 
P(Send | S en d,next) 1. 

We now describe p for the box S X1 other boxes being similar. For all a, b £ 
{it ,X!,Zi,^Zi | z £ {x,y},i £ {2,..., n}}, p^ 1 ’ 1 I sZ 1 ’ 1 , next) = p{s | 
Sa 1,1 .next) = l/(4n+ 1) and for all c, d £ {(J, b, Zj, -iZj | z £ {x, y}, i £ {1,..., n}}, 
pisj 1 ’ 2 | Sc 1 ’ 2 , next) = l/(4n + 2). 

• 0(s en rf) = b, O(s^) = a and 0(s a ) = a when a is a Boolean variable or its negation 
and for all other state s, 0(s) = ft. 

The initial distribution po is l Sinit and the set of secret states is Sec = {s en d}- 

We show that </> is t rue iff th e disclosure of M for observations of length 2(n + m ) + 3 
is greater than or equal to ^. First observe that for any strategy, the measure of runs 
reaching state s en d with observation of length 2(?z + m) + 3 is exactly ^r. Indeed, during 
each of the first 2 n actions, whatever the choices of the strategy, there is a probability 
^ to go in one of the boxes and ^ to advance to the next choice, thus a probability 
to reach the state sj. From there every run reaches s en d in 2 {m + 1) steps. If the 
strategy is such that some variable and its negation are read on the way to s en d, then 
there exists a run with same observation reaching the second part of a box where every 
observation can be triggered, and thus the run reaching s en d will not disclose the secret. 
Intuitively, during the first 2 n steps, every Boolean variable is assigned a value: 
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either chosen by the strategy as it chooses whether Xi or —i Xi occurs in the observation 
for all 1 < i < n, or randomly as m and ~>yi both have equal chance of being triggered. 
During the last 2m steps, the strategy must trigger a Boolean variable in every clause 
of the disjunction so that if a clause is not satisfied by the current assignment, then a 
Boolean variable is observed as both true and false during the run. Thus the observation 
would not disclose the secret. In order for a measure of of runs to disclose the secret, 
for every assignment of the y, the controller must force the run reaching s en d to disclose 
the secret. 

Suppose that (j) is equivalent to true. Thus there exist functions ( fi)i=i... n (expressing 
the choices for x±,..., x n ) such that for every set of assignments (ai,..., a n ) of the vari¬ 
ables yi,...,y n the Boolean formula -0[/i (), «i, f 2 ( 0 , 1 ),f n (a 1 ,..., a n _i), a n ] is true. 
We choose a strategy a such that for every possible set of assignments (ai,..., a n ) for the 
variables y 1 ,...,y n , for all i G {0,... n — 1}, 0-(tt/i()ai/ 2 (ai) ■ ■ • a i) = fi+i(oi, ■ ■ •, a»). 
Moreover for G {1,2,3}, there exists z k+ G |/i(), ai, / 2 (ai),..., a n } 

such that cr(jj/i()ai/ 2 (ai).. .a n %zi ii Z 2 h . ..Zfc ife ) = z k + i< . The choice of the strat¬ 
egy is arbitrary in the other cases. Such a strategy can be defined since the formula 
-i/>[/i(), ai, / 2 (ai),..., f n (oi, ■ ■ ■, a n _i), a n \ is true and thus every clause is satisfied by 
this choice of assignments. 

With this strategy, the fixed-horizon disclosure in 2 (n + m + 1) steps is ^. In other 
words, all the runs reaching the secret disclose it. Indeed let p be a secret run of length 
2 (n + m + 1). There exists an assignment a\,...,a n G {yi,~ i 2/i, ■ ■ • y n , ~ 1 y n } such that 
0 (p) = U/i()ai/ 2 (ai).. .a n Uzi n z 2i2 ...z mi J>. By choice of a, if, for z G {x,y} and 
i G {1, • • • n}, Zi appears in the observation of p , -1 z* does not appear, and vice versa. 
Therefore as b can be read either in s en d or in a state reachable only by runs observing 
a Boolean variable and its negation, p discloses the secret. 

Conversely, suppose that 4> is n °t equivalent to true and let a be a strategy, which 
can be assumed to be deterministic thanks to Proposition |7.3| We build partial functions 
fi : £ 2 * 1 —> Act consistent with er: for every observation j \w G j \T? 1 of some run p, if o 
chooses action a G A(last(/o)) for p (i.e., a(p){a) = 1) then fi(w) = a. As 4> is not 
equivalent to true, there exists an assignment (ai,...,a n ) for the variables y\,...,y n 
such that the Boolean formula ai, / 2 (a 1 ),..., f n (a 1 ,..., a n _i), a n ] is false. We 

now build a run with non null probability, reaching the secret but not disclosing it. 

By construction, there exists p such that 0 (p) = U/i()ai/ 2 (ai)... f n (oi ■ ■ ■ a n _i)a n jj, 
with last(/o) = S(j and faip) > 0 (where again F a stands for Pm ct ( mo ))- L e ^ * £ N 
be an integer such that the clause zq V Zj 2 V Zj 3 is not true under the assignment 
[/i(), ai, / 2 (ai), • ■ •, f n (oi, ■ ■ ■, a n -i), a n ], in other words such that the negations of 
Zi ,, Zi 2 and Z{ 3 were chosen as assignment. Let p' be the run of length 2(n + m) + 2 ex¬ 
tending p and ending in s en d■ Then p' does not disclose the secret: indeed, there exists 
j G {1, 2,3} such that z* . appears in its last 2m observations while its negation (written 
-1 Zj. here) appears in the first 2n + 1 observations. Thus there exists a run with same 
observation leading to the second part of the box S-, Zi , which is outside the secret and 
where every observation is possible. As the total measure of runs reaching the secret 
is 25 s an d at least a subset of measure P CT (p) of the runs reaching the secret do not 
disclose it, the disclosure of M is strictly smaller than hi 2(n + m) + 3 observation 
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steps. □ 

The existence of an optimal strategy in the first part of the proof implies that 
the limit-sure and the almost-sure problem are equivalent. Moreover, the secret being 
revealed with probability 1 in a given number of steps, every run must reach the secret 
in this number of steps. Testing if there exists a strategy such that every run reaches a 
set of target states in a given number of steps in an OMDP can be solved in polynomial 
time. 

Remark 7.1. The proof of hardness can be adapted for maximal fixed-horizon e-disclosure, 
but the algorithm for membership cannot be directly applied. The e-disclosiLre could 
however be computed by maximising an exponential system of equations, resulting in an 
exponential time algorithm. 


4.2 Minimal disclosure 


The proof of the next theorem is similar to the proof of Theorem |7.7| on the fixed-horizon 
maximal disclosure. 

The hardness result is obtained once again using a reduction from the validity of a 


QBF. Many of the ideas used in the proof of Theorem 7.7 reappears here: the run is still 
composed of two parts, in the first one it gives an assignment to the Boolean variables 
and in the second one the strategies goes through the clauses of the formula and verify 
it can satisfy them. We however give the full proof due to non-negligible differences. 
Now the strategy must satisfy every clause of the formula in order for the run not to 
disclose the secret. 

For the strategy decision problem, contrary to the maximisation case, due to the 
randomisation, there does not necessarily exists an optimal strategy. In order to get the 
same complexity for the strategy decision problem, we establish that when a randomised 
decision rule must be selected in the optimal strategy, it can always be uniformly dis¬ 
tributed over its support. 

Theorem 7.8. The fixed-horizon minimal value is computable in PSPACE. The fixed- 
horizon minimal disclosiLre problem is PSPACE -complete. In addition, the strategy de¬ 
cision problem is also decidable in PSPACE. 

Proof. The procedures for the first two problems are very similar to the ones used in 


Theorem 7.7 There are only two differences. First, given B{ the current belief and Oj+i 
one computes Bi+\ = NextMax(R,, Oj+i) (independently of the guessed actions a^ s ). 
Second, the computation procedure operates by decreasing values of i when the value 
is less or equal than 

In order to decide whether a strategy exists that provides the minimal value, one 
guesses this strategy in PSPACE as before. However there is an additional difficulty since 
the (possible) optimal strategy may be randomised. Thus during the procedure, given 
some belief B and some state s, one guesses the support A' C A s of the decision rule 
and one defines the decision rule say <5 as a uniform choice over A'. We claim that this 
restriction is sound. Assume another decision rule 5' with same support would provide 
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a smaller value. Then, since the support are unchanged, the decision rule informally 
described as (l + e)5' — e5 for small enough e would still provide a better value, meaning 
that the support A! cannot be used to find an optimal strategy. 



Figure 7.13: Reduction of the validity problem to the disclosure on a fixed horizon. The 


box S X1 is represented in Figure 7.14 


Like for the case of maximisation, the hardness of the fixed-horizon minimal dis¬ 
closure problem is obtained by a reduction from the validity of a quantified Boolean 
formula. Let cf) = 3x\\/yi3x2 ■ ■. Vy n ^ with if; = A;=i m ( z h V Zi 2 V Zi 3 ) a closed QBF 
where we assume w.l.o.g. that in every clause the literals are distinct. We build the 
OMDP M = (S, Act,p, 0) (represented in Figure 7.13) where: 

• S = {Sinit, Send, S))} U {s Zi \ z G {x,y},i = 1. .. n} U {s^ 


2 e {x,y},i = 


l...ra}U{sJ | i G {1,..., m}} U {s ''■> \ i G G {1, 2,3}} U | z G 

{x, y},i = 1... n} Uj e |x ; n i (S Xi OS Vi L)S^ Xi OS^ Vi ) where for z t one of the Boolean 
variable, S Zi = {s% \ a G {#, z i: ^Zi \ z G {x,y},z G {1,. .., n}}} U {s^, s^}. 
Similar for the box S-, z . of the negation of a variable. 


• A(si n it) = {xi,-ixi}, and for all i < n, A(s Vi ) = A(s^ yi ) = {x i+ i,-ix i+ i}. For 
i G {1,..., m}, A(sl) = {1, 2, 3} and for every other state (even in boxes), 7l(s) = 
{next}. 
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4n+2 


Figure 7.14: Representation of the box S Xl 


(with the conventions of Figure 7.8). 


• p(s a | s ini t,a ) = p{sa l \ s iniu a ) = 1/2. For all i < n, p{s a \ s yi ,a) = pisa’ 1 \ 
s yi ,a) = p(s a | s ^ Vi , a) = p(sa 1 \ s^ Vi ,a) = 1/2. For all i < n, p{s Vi | s Xi ,next) = 
Pisyi 1 I s *i > n ext) = p(s^ Vi | s Xi ,next) = p{s^4 | s Xi ,next) = p(s Sy .^ x .,yi) = 

P(4i 4 I s ^xi,yi) = p{s^ yi | s-,Xj) next) = p(s4r\ | s^next) = 1/4. p{s\ \ 

sj, next) = 1. For all i = 1... m, j £ {1,2,3}, p{s z 'i | s\ , j ) = 1, and if i < m, 
p(s\ +1 | sW,next) = 1. Finally p{s en d \ s Zm i , next) = 1, and for all z £ {x,y}, 
i = l...n, p{s^ nd | s end , next) = 1/(2 n). 

We now describe p for the box S Xl other boxes being similar. For all a £ 
(t),xi ,Zi,-iZi | 2 £ {x,y},i G {2 ,...,n}},b G (tt, b, z*, -'Zi | z G {x,y},z G 

{l,...,?r}} p(a^ | s * 1 , next) = l/(4n + 2), and p(s%' Xi I s^next) = p(sf \ 

s Xl , next) = p(sy | Sj 1 , next) = 1. 


• 0(s en d) = b, O(s^) = a and 0(s“) = a when a is a Boolean variable, its negation 
or b. For i = 1.. . n, 0(s^ d ) = O(s^) = 0(sJ Xl ) = 2i — 1 and 0{s v J nd ) = O(s^) = 
0 = 2 i, and for any other state s, 0(s) = J). 

The initial distribution po is 1 Sinit and the secret runs are the ones visiting s en d (Sec = 
{send} U {s z ^ nd \ z e {x,y},i = 1.. .n}). 

In a similar fashion as what was done in the hardness part of the proof of Propo¬ 


sition 7.7 we show that </> is true iff the disclosure of M is equal to 0 in 2 (to + m + 2) 


steps. First observe that a run p reaching s en .d can be extended for all j G {1,..., 2n} 
in a run pj such that O(pj) = 0(p)j. Moreover, p\ discloses the secret iff x\ and —>x\ 
both occur in 0(pi) (and similarly for the other Pjs). Indeed a run reaching S Xl or S-, Xl 
cannot have triggered both observations x\ and —>x\ and also end with observation 1. 

Intuitively, during the first 2 n steps, all Boolean variables are be assigned a value: 
either chosen by the strategy as it chooses whether Xi or —>Xi occurs in the observation 
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for each 1 < i < n, or randomly as y* and -ij/j both have half a chance of being triggered. 
During the last 2m+1 steps, the strategy must choose a Boolean formula in every clause 
so that if a clause is not satisfied by the current assignment, then a Boolean variable is 
observed as both true and false during the run. The last step then triggers randomly 
the observation j for j G {1,... 2n}. 

Suppose that (j) is equivalent to true. Then there exist functions {fi)i= i... n such 
that for every set of assignments (ai,..., a n ) for the variables y \,..., y n the Boolean 
formula ip[fi(), a\, / 2 (ai),..., f n (a i,..., a„_i), a n )] is true. We choose a strategy a 
such that for every possible set of assignments (ai,..., a n ) for the variables y i,.. •, y n , 
and for all i, 0 < i < n — 1, cr(tt/i()ai/ 2 (ai)... at) = /j+i(ai,..., a,). Moreover for 
k G {l,...,m}and*i,...,4 G {1,2,3}, there exists Zk+i ik+1 £ {/i(), ai, / 2 (ai),..., a n } 
such that o’(tt/i()ai/ 2 (ai)... OnWzi^ z 2i2 ■.. z kik ) = ^fc+i ife+1 ■ The choice of the strategy 
is arbitrary in the other cases. Since ai, / 2 (ai),..., f n (a i,..., a n _i), a n )] is true, 

every clause is satisfied by this choice of assignments, hence it is possible to define such 
a strategy. 

With this strategy, the fixed-horizon disclosure in 2{n + m + 2) steps is 0. In 
other words, none of the runs reaching the secret discloses it. Indeed let p be a secret 
run of length 2(n + m + 2), then there exist oi,..., a n G {yi, —>2/i,... y n , -iy n } and 
j G {1,... ,2n} such that 0 (p) = tt/i()ai/ 2 (ai)...a n ttt)M n z 2i2 • ..z mim \>j. By choice of 
a, if, for z G {x, y} and i G {1,... n}, z% appears in the observation of p. -i Zi does not, 
and vice versa. Therefore as b j can be read either from s en d or in a box state outside 
of the secret reachable only by runs that do not observe a Boolean variable and its 
negation, p does not disclose the secret. 

Conversely, suppose that <f> is not equivalent to true and let a be an arbitrary 
strategy. We first build a deterministic strategy a' with smaller or equal disclosure. 
The first choice concerns {.Ti,-a’i} and the next observation in a run corresponds to 
that choice. Consider o\ (resp. o^) the strategy that selects x\ (resp. -laq) and then 
plays like a. Due to the fact that observations are distinct, the disclosure value w.r.t. <j 
is a convex combination of the ones of o\ and a[. So one substitutes a by the one with 
smaller or equal disclosure. A similar pattern applies for every choice until reaching the 
horizon. Thus by iterating this transformation we obtain a deterministic strategy. So 
we assume now that a is deterministic. Since there is a finite number of such strategies 
for fixed horizon, it only remains to prove that the disclosure value under a is positive. 
We build partial functions /,; : S 2 * i —> Act consistent with a: for every observation 
ft w G tJS 2 * of some run p , if a chooses action a G M(last(p)) for p. then we set fi(w) = a. 
As 4> is n °t equivalent to true, there exists an assignment (ai,... ,a n ) for the variables 
yi,---,y n such that the Boolean formula ip[f- i(), ai, / 2 (oi), • • •, f n {ai, ■ • •, a n -i,a n )\ is 
false. 

We now build a run disclosing the secret. By construction, there exists p such that 
0 (p) = tt/i()ai/ 2 (ai)... f n (a i... a n _i)a n tt, leading to last(p) = s # with P a (p) > 0. Let 
i G {1,..., m} such that the negations of zp , Zi 2 and z l3 were chosen as assignment hence 
Zi l V z i2 V z i3 is not true under the assignment [/i(), ai, / 2 (ai f n (ai, ■■■, a„_i), a n )]. 
Let p' be a run of length 2 (n + m) + 3 extending p and ending in s en d. Then p' does 
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not disclose the secret because there exists j € {1,2,3} such that Zi j appears in the 
previous 2m observations while its negation (written -> z\ i here) appears in the hrst 
2n+1 observations. Let p" of length 2(n + m + 2) extending p' by ending in s^ d . There 
is no other run with the same observation and p" is a secret run, thus p" discloses the 
secret. Therefore the disclosure of M is positive. 

Observe that this reduction also works for finite horizon since no further disclosure 
may occur after the hrst occurrence of a state in {s*^ d , ..., s*™ d , s y J nd , ..., s y e ™ d }. □ 

Contrary to the case of maximisation, the above proof implies PSPACE-completeness 
for the limit-sure and almost-sure problem for disclosure minimisation. 

Remark 7.2. As for maximisation, the proof of hardness can be adapted for e-disclosure 
and the algorithm for membership cannot be directly applied. The minimal fixed hori¬ 
zon e-disclosure could however be computed by minimising an exponential system of 
equations, resulting in an exponential time algorithm. 


5 Conclusion 

To our knowledge, the opacity of probabilistic systems had only been studied in order 
to maximise the disclosure of the system. Moreover, these studies always restricted the 
framework so that the strategy that is chosen does not modify if an observed sequence 
is disclosing or not, leaving the general case open. In the context of the previous 
studies, only maximisation was considered, which is understandable as maximisation 
and minimisation of disclosure are similar: they both consists in the optimisation of 
a fixed event. We, however, focused on the general case, both for maximisation, and 
for minimisation. In our framework, maximisation and minimisation present a strong 
asymmetry. Indeed, when considering finite horizon, most maximisation problems are 
undecidable although deterministic strategies are optimal. In contrast, minimisations 
problems are decidable, but good strategies often require randomisation. Note that 
a complexity gap (PSPACE-hard versus in EXPTIME) remains to fill for the finite- 
horizon minimisation problem. For fixed horizon, there is still an asymmetry between 
maximisation and minimisation that clearly appears in some parts of the proofs. But 
it is not as strong as in finite horizon and algorithms with good complexities can be 
obtained for both. 

Although we used a variant of Markov decision processes enriched with observation) 
to represent our models, opacity is not an usual MDP problem. Indeed, opacity is an 
hyper property as the disclosure depends on a set of paths linked by their observation. 
This gives a partial observation flavour to opacity. Opacity as seen here is therefore 
a problem in between OMDP and POMDP. For this kind of problems, as seen in this 
chapter, it is important to determine whether the problem can be translated to an 
MDP or a POMDP problem in order to use the results known on these models. Here, 
maximisation of the disclosure was closer to POMDP problems while minimisation was 
closer to OMDP problems. 



234 


Opacity 


A promising research direction to consider is the approximate notion of opacity. It 
is the most natural notion of opacity. Indeed, if an attacker knows there is a 99% chance 
to be in the secret, the secret could be considered to be disclosed. As we have shown in 
Section [l] the most naive definition of approximate opacity is undecidable even with¬ 
out control. For diagnosability, we showed in Chapter [I] that AFF-diagnosability, an 
elaborate notion of approximate diagnosability was decidable for passive systems. A 
natural question is thus to determine if we can define a similar notion of opacity. A 
notion that would measure the set of infinite secret runs which disclose with arbitrarily 
high probability the secret for example. As it is close to AFF-diagnosability, we con¬ 
jecture that it should be decidable for observable Markov chains. However, in active 
system, the finite-horizon maximisation/minimisation disclosure problem are likely to 
be undecidable. 


Chapter 8 

Conclusion 


Contributions 

This thesis constitutes part of the work towards a theoretical analysis of partial obser¬ 
vation problems in a stochastic framework. More specifically it focused on the problem 
of diagnosis. Diagnosis had already been studied for stochastic systems |TT0511CK131 
1BFH+14) . however the definitions used varied and many central issues had been left 
open. The first step to set solid foundations for the analysis of diagnosis in probabilis¬ 
tic systems was thus to define precise and realistic notions of diagnosis which would 
encompass the ones already established. This was done in Chapter [2] Before focus¬ 
ing on any specific framework, we performed in Chapter [3] a semantical analysis of the 
different notions of diagnosability. While some intuitions on the relations between the 
notions could be obtained directly from the definitions, the analysis allowed, among 
other things, to establish formally these links, with a few surprises due mostly to the 
distinction between finite systems, finitely-branching systems and infinitely-branching 
systems. 

We then turned to multiple specific frameworks and developed methods to decide 
diagnosability with optimal complexity. First, we focused on passive systems. More¬ 
over, in Chapter [IJ we restricted ourselves to finite systems. This important restriction 
pushed us to make once again some semantical analysis in order to obtain refined re¬ 
sults exploiting the finite number of states. This gave us precise characterisations of 
the decidable notions of diagnosis, allowing us to establish the exact complexities of 
the problems. We also showed how to automatically build diagnosers associated with 
each notion of diagnosability. In Chapter [5j we extended our analysis to infinite-state 
systems. This immediately raised one important issue that we did not have for finite 
systems: how to represent such systems. In consequence, we studied several possible 
representations, one of which yielded multiple decidability results. These decidability 
results were obtained in large part thanks to the analysis made in Chapter [3] This 
emphasizes the importance of a good understanding of a notion and how to characterise 
it as a preamble to study the problem. 

We then considered active systems. As for stochastic infinite systems, many different 
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But contrary to stochastic infinite systems, diagnosis had 

BFH + 14] , providing a framework 


frameworks may be studied 

already been studied for stochastic active systems 
our developments. The latter revealed an issue with the control of a system: ensur¬ 
ing diagnosability could be at the expense of the correct performance of the system. 
We studied in Chapter [6] how to limit the degradation of the system while preserving 
diagnosability. More precisely, we defined notions of the degradation of a system and 
showed the decidability and precise complexity for some of them, and established the 
undecidability of the others. 

In the last chapter, Chapter [7J we switched our focus to opacity, another partial 
observation problem. In active systems, we showed how, when possible, one could 
develop strategies maximising or minimising the opacity of a system. The main element 
that made this analysis successful is the understanding of the forms that the optimal 
strategies would take. 

Our contributions, while providing a good foundation for the diagnosis of proba¬ 
bilistic systems and many interesting results, are far from giving the whole picture. In 
the next section we provide a list of remaining open questions and research directions 
extending the thesis. 


Perspectives 

The current thesis opens quite a few perspectives, some of which were already given 
in each chapter conclusion and are partially repeated here. We classify these ideas 
depending on whether they are short-term, mid-term or long-term objectives. This 
decomposition represents how direct the link between the current work and the per¬ 
spective is. We start with the short-term perspectives, i.e. the problems immediately 
raised by the works presented here. 

• The most immediate perspectives are the ones given by the gaps within our results: 
notions for which we could not establish the decidability status, complexities that 
are not tight, etc. For example, the algorithms given to decide the exact notions 
of diagnosability in probabilistic visibly pushdown automata are in EXPSPACE 
while the proven lower bound is only EXPTIME. Another open question is the 
exact complexity of computing the minimal disclosure of the opacity (PSPACE 
lower bound versus EXPTIME upper bound). The main open question however 
is the decidability status of the FA-diagnosability in pVPA. We showed that this 
notion was harder than the other notion of exact diagnosability by studying its 
membership in the Borel hierarchy, but could not give an algorithm nor an unde¬ 
cidability proof. We conjecture that this notion is decidable and, in fact, with an 
algorithm of the same complexity than the other. Indeed, the proofs of the the 
non-expressivity results that limited the study of FA-diagnosability uses systems 
that cannot be expressed by pVPA. There could exists a pathL formula that would 
characterise FA-diagnosability when restricted to pVPA. An ongoing work seems 
to confirm this to be true. 
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• Another immediate perspective is raised by the introduction of the pathL logic. 
It was used in Chapter [5] in order to decide some notions of exact diagnosability. 
This logic may be useful for example to test properties such as the uniformity of 
the speed of diagnosis, the boundedness of the mean detection time of a fault (or 
mean time before an information about the correctness of a run), etc. Moreover, 
if the pathL logic cannot be used directly, one could define an enriched version of 
this notion with greater expressive power. This enriched version must be carefully 
designed so that the generated formulae can be checked. 

• The last immediate perspective relates to the active framework. Whether it be 
for diagnosis or opacity, we only focused on exact notions. It would be natural to 
tackle the, usually harder, approximate notions. This is in fact an ongoing work. 
The current results seem to point toward decidability for AFF-diagnosability in 
active systems while similar approximate notions are undecidable for opacity, both 
for maximisation and minimisation. 

We now turn to mid-term perspectives. They correspond to problems that are 
strongly connected to this thesis, while not being immediate. 

• In our active framework, the observations are clearly given by the model. This 
represents in reality sensors within the system. Using a sensor has a cost. There¬ 
fore, instead of having fixed sensors, one could have a list of potential sensors 
associated with costs. The goal would then be to obtain diagnosability while 
minimising the cost. A cost could also be given to having the sensor turned on, 
forcing the optimal strategy to decide when it needs to have the sensor operating. 
Some works were already done on this subject, see (CTOBlj and (TT07| . 

• Faults, as defined in this document, are a boolean property: a system is either 
faulty or correct. Moreover, they are permanent. Once a fault occurred, we 
did not consider as important to decide if more faults would be created later for 
example. No matter the number of faults, the run is deemed faulty. One could 
envision a different idea for the fault. A fault could represent a partial degradation 
of the system, the failure of one of its non-vital component or something that can 
be repaired (see |FHLM18| for a study of repairable faults in a non-stochastic 
framework). Seen this way, many new questions arise. In passive systems, this 
means defining measures of correctness for a system and testing properties, for 
example on the delay of the fault counter (one can test if this delay is bounded, 
if it is unbounded but in o(n) where n is the length of the run, in 0(n)...). 
In active systems, we would wish to find controllers that optimise the measures 
of interests or that ensure good delays of detection for the fault counter. This 
would obviously only be an interesting study for systems where many faults will 
be triggered. However, this is not an unrealistic assumption as every system is 
slowly degraded due to time elapsing. 

• Our diagnosability algorithms currently only gives a Boolean answer. In order for 
system designers to modify its system so that it becomes diagnosable for example, 
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they need to know what is the cause of the missed fault. As a consequence, it 
would be interesting to design algorithm able to give a counter-example to the 
diagnosability of the system whenever it exists. This notion of counter-example 
needs to be made precise. In an LTS, a counter-example can be given by an 
ambiguous cycle of the system. As with probabilistic systems, this cycle may 
have a zero probability, the counter-example must thus be able to describe a set 
of runs with positive probability. For example, by giving a finite faulty run such 
that any extension of it is ambiguous. However, this kind of counter-example 
does not necessarily exist, in pushdown systems for example. The generation of 
counter-examples in stochastic systems has been studied, see |ABD~tl4], but for 
different objectives than diagnosability. 


• The main formalism we chose, probabilistic labelled transition systems, has its 
limitations. One could be interested in studying higher-level models such as 
stochastic Petri nets, stochastic process algebra (with PEPA for example pa 
Chapter 3]), etc. can be more appropriate to represent some real life systems. 
Indeed, high-level formalisms usually are (often exponentially) more concise than 
low level formalisms, hence a greater comfort for designers. Moreover, high-level 
formalisms usually have a structure, which allows to conceive more efficient al¬ 
gorithms as the generated systems (pLTS, MDP, etc.) benefit from additional 
properties. For example, when a system is a synchronised product of several com¬ 
ponents, the transition matrix or the infinitesimal generator can be computed 
using tensor products of the matrix representing the different components (see 

e.g. (HM9511HM963). 

• During this thesis, we designed many algorithms. In parallel to the previous 
item, it would be useful to implement these algorithms. This tool could then 
be used to solve the diagnosability problems for pLTS or for some higher-level 
models whose semantics is an appropriate pLTS. This implementation should be 
integrated within an existing tool to benefit from the possibilities offered while 
enriching it. A good candidate would be COSMOS [BBD +15] , a statistical model 
checker for the hybrid automata stochastic logic. 

We now end with the long-term perspectives, more remote to our current work. 


• We established many results within this thesis. Some of which used methods 
that are quite usual, some required to use new ideas such as the pathL logic. 
These new ideas may be useful in order to tackle other issues related to partial 
observation such as identification problems, stochastic games... It could therefore 
be interesting to see which kind of problems, in these other frameworks, would 
benefit from our approach. 

• Another direction comes from another interpretation of faults. Let us proceed 
through an example. When someone is sick, multiple symptoms appear or do 
not appear in the body. These symptoms each correspond to a failure of the 
human body, so each are a fault that must be detected. However, in order to 
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cure the patient one needs to link the patterns to a common cause: an illness. 
In other words, one wants to deduce a meta-information from the behaviour of 
the system. Finding the origin of the fault has been studied under the term 
causality, but, as explained in |GSS17j . this approach focused on static systems. 
In dynamic systems, this meta-information may be seen as a pattern that must 
be detected [JMPC06] . This objective can be seen as an extension of diagnosis, 
but it goes into the long-term category as it corresponds to what seems to be a 
far more general question. It can also be linked to questions of identification of 
complex behaviour, possibly similar to what was studied in [Pie m- 

• Diagnosis is a research domain with clear applications. As a consequence, it would 
gain a lot from being studied in cooperation with the industry. This would allow 
researchers to better understand the industry’s need. For example, there could 
exists definitions of diagnosability that we did not focus on, yet have relevance 
from an industrial point of view. Moreover, while our methods are efficient in 
theory, they may raise practical issues that we did not consider. There are already 
some cooperations such as |HF12] where the authors investigate the problem of 
building a model appropriate for diagnosis out of a real system, in their case, a 
network. 

• The last perspective discussed here is of a different nature: it is not a research 
direction. However, it could still have a great impact on the domain of research. 
There exist many contributions which either establish a known result or have an 
erroneous proof for a theorem that was already proven false (trying to give a 
PTIME algorithm for a PS PACE-hard problem for example). This clearly shows 
the difficulty for researchers to know the current state of the research, even for a 
specific domain such as diagnosis. To tackle this issue, one possibility may be to 
try to build a cooperative website that would gather all the results on diagnosis, 
something in the spirit of the POMDP webpage |POM| . This way, one could 
efficiently find the current state of the art on the domain and this would save a 
lot of useful time to many researchers. This obviously has some issues. To be 
useful, the existence of such a website has to be spread and people must keep it 
up to date. It would also raise many questions of organisation due to the width 
of the domain, even when restricted to diagnosis (the many existing frameworks, 
diagnosability notions, methods of approach,...). The various surveys on diagnosis 
issues [ZL131 fB as!4j would help dealing with this point. But most of all, building 
such a website is extremely time-consuming. 
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dir avec la multiplication des systemes 
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Abstract : The control of the informa¬ 
tion given by a system has recently seen 
increasing importance due to the omni¬ 
presence of communicating systems, the 
need for privacy, etc. This control can 
be used in order to disclose an informa¬ 
tion of the system, or, oppositely, to hide 
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thin it or not. In this PhD, we study the 
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